[RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

The "beyond page table accessed-bit DAMON extension" project [1]
milestone 2 is for building the basic infrastructure for continued
extension.  After starting it, I studied perf event little and found a
few demons in details.  I therefore spent time on high level design.  As
it started looks like making sense to me, I'd like to share the humble
design first and get feedback from more eyeballs.

Add a blueprint of the milestone 2 as a high level design document.
Describe the API and ABI of data attribute report infrastructure, and an
example use case of it for per-CPUs write-only monitoring.  The API and
ABI will later be extended for more data attributes reporters utilizing
AMD IBS, Intel PEBS, Arm SPEs like h/w features and any possible
DAMON-external reporter kernel components.

Request for comments
====================

This is an RFC for high level design discussion, so no implementation is
provided.  Comments I want to receive include but not limited to below.

Does the idea for perf CPU cycle sampling based access monitoring makes
sense?

Does the design provides clear idea about how it can be extended for the
future extensions that already scoped for milestone 3 (AMD IBS, Intel
PEBS, Arm SPE)?

For the foreseeable extensions (AMD IBS, Intel PEBS, Arm SPE), would the
required change minimum and maintainable?

Next steps
==========

I will start implementing the infrastructure and perf CPU cycle sampling
based access monitoring as described in this patch.  It will go through
the normal development process.  Delivery of it will be the completion
of the milestone 2.  As set on the initial roadmap [1], ETA of the
delivery is LSFMMBPF'27.  No delay is expected so far.

[1] https://lore.kernel.org/all/20260525225208.1179-1-sj@kernel.org/

Signed-off-by: SJ Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 44 +++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index a465e34bb1d9c..1e699e3f8a496 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -433,6 +433,50 @@ monitoring.
 Refer to the :ref:`admin guide <damon_usage_sysfs_probes>` to know how users
 can use the mode.
 
+Data Attributes Reporting
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Infrastructure for receiving attribute information from DAMON-external kernel
+components.
+
+Because DAMON is not hooked in every place but runs in its own worker thread,
+it cannot retrieve some types of data attribute information by itself.  Many of
+those are available to DAMON-external kernel components such as perf event
+callbacks and page fault handler.  For such cases, DAMON provides attributes
+reporting infrastructure.  Using its API, any kernel components can report
+their finding to DAMON.
+
+The API for doing the report is a function, damon_report_attrs().  The function
+receives the attribute information and saves it in DAMON's internal buffer.
+The function is safe to be called in any context including NMI.  The internal
+buffer has a fixed size and keeps only latest reports of the size.  Because one
+buffer size doesn't fit all, multiple report buffers of different size and
+rules can be generated and shared by multiple reporters.  DAMON provides API
+for such management of report buffers.  damon_report_attrs() can specify into
+which buffer the report should be saved.  DAMON user interface (ABI) for doing
+the buffer setup, and enable/disable reports of supported types is also
+supported.
+
+The data attribute probe supports the reports.  The probes can be configured to
+hit for a given memory region depending on the existence of specific types of
+reports that associated with the memory region in given report buffers.
+
+For example, per-CPUs write-only monitoring can be configured using this
+infrastructure like below.
+
+The user sets a report buffer and enable CPU instruction sampling report for
+the buffer.  DAMON will enable perf event counter for CPU cycles sampling.  The
+sampling callback will save the sampling time CPU instruction and the id of the
+CPU that was executing it, into the DAMON report buffer, using
+damon_report_attrs().  DAMON can automatically adjust the perf event sampling
+frequency to minimize old report drops due to the buffer size limit.
+
+The user further sets a DAMON probe.  The probe hits for a given DAMON region
+if a store instruction for a memory in the region that was executed from a
+given CPUs group exists in the report buffer.  As a result, the probe hit
+counts represent relative frequency of writes from a given CPUs group to a
+given memory region.
+
 Dynamic Target Space Updates Handling
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

base-commit: 366c6418699768dba5826d412700a29d562ca039
-- 
2.47.3

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Docs/mm/damon: CPU cycle instruction sampling lacks target memory address
--

commit a164f9ff38c8cd1963db3e0c7ea062dbc93135a3
Author: SJ Park <sj@kernel.org>

Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

This patch documents the high-level design for DAMON's data attributes
reporting infrastructure, including the damon_report_attrs() API and an
example use case for per-CPU write-only monitoring using perf CPU cycles
sampling.

> diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
> index a465e34bb1d9c..1e699e3f8a496 100644
> --- a/Documentation/mm/damon/design.rst
> +++ b/Documentation/mm/damon/design.rst
> @@ -433,6 +433,50 @@ monitoring.
>  Refer to the :ref:`admin guide <damon_usage_sysfs_probes>` to know how users
>  can use the mode.
>  
> +Data Attributes Reporting
> +~~~~~~~~~~~~~~~~~~~~~~~~~

[ ... ]

> +For example, per-CPUs write-only monitoring can be configured using this
> +infrastructure like below.
> +
> +The user sets a report buffer and enable CPU instruction sampling report for
> +the buffer.  DAMON will enable perf event counter for CPU cycles sampling.  The
> +sampling callback will save the sampling time CPU instruction and the id of the
> +CPU that was executing it, into the DAMON report buffer, using
> +damon_report_attrs().  DAMON can automatically adjust the perf event sampling
> +frequency to minimize old report drops due to the buffer size limit.
> +
> +The user further sets a DAMON probe.  The probe hits for a given DAMON region
> +if a store instruction for a memory in the region that was executed from a
> +given CPUs group exists in the report buffer.  As a result, the probe hit
> +counts represent relative frequency of writes from a given CPUs group to a
> +given memory region.

[Severity: Medium]
In the Data Attributes Reporting section of Documentation/mm/damon/design.rst,
how can the DAMON probe determine whether a sampled store instruction targeted
a given DAMON memory region if the callback only saves the CPU instruction and
CPU ID from a generic CPU cycles counter?

A sampled instruction (or instruction pointer) only encodes the opcode and
register operands, not the effective virtual or physical data address being
written. Saving only the instruction and executing CPU ID via
damon_report_attrs() does not appear to provide the target memory address
needed to match against a DAMON region when the probe inspects the buffer.

Even if the perf overflow callback attempted to resolve the target data
address at NMI time, generic CPU cycle counter interrupts
(PERF_COUNT_HW_CPU_CYCLES) are non-precise on out-of-order CPUs and suffer
from skid. At NMI time, regs->ip typically points to a subsequent instruction
rather than the instruction that caused the overflow, and the register values
in struct pt_regs may already have been modified.

Furthermore, without hardware data-address sampling features (such as Intel
PEBS, AMD IBS, or Arm SPE populating perf_sample_data->addr or phys_addr,
which are scoped for Milestone 3), resolving the effective address from a raw
instruction would require fetching instruction bytes in NMI context, running
an architecture-specific instruction decoder and address-calculation emulator,
and translating virtual addresses to physical addresses for paddr monitoring.

How is the target memory address intended to be obtained and matched to DAMON
regions in Milestone 2 with generic CPU cycle sampling?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261017195747.89961-1-sj@kernel.org?part=1

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

On Sun, 18 Oct 2026 19:58:01 +0000 sashiko-bot@kernel.org wrote:

> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] Docs/mm/damon: CPU cycle instruction sampling lacks target memory address
> --
> 
> commit a164f9ff38c8cd1963db3e0c7ea062dbc93135a3
> Author: SJ Park <sj@kernel.org>
> 
> Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)
> 
> This patch documents the high-level design for DAMON's data attributes
> reporting infrastructure, including the damon_report_attrs() API and an
> example use case for per-CPU write-only monitoring using perf CPU cycles
> sampling.
> 
> > diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
> > index a465e34bb1d9c..1e699e3f8a496 100644
> > --- a/Documentation/mm/damon/design.rst
> > +++ b/Documentation/mm/damon/design.rst
> > @@ -433,6 +433,50 @@ monitoring.
> >  Refer to the :ref:`admin guide <damon_usage_sysfs_probes>` to know how users
> >  can use the mode.
> >  
> > +Data Attributes Reporting
> > +~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> [ ... ]
> 
> > +For example, per-CPUs write-only monitoring can be configured using this
> > +infrastructure like below.
> > +
> > +The user sets a report buffer and enable CPU instruction sampling report for
> > +the buffer.  DAMON will enable perf event counter for CPU cycles sampling.  The
> > +sampling callback will save the sampling time CPU instruction and the id of the
> > +CPU that was executing it, into the DAMON report buffer, using
> > +damon_report_attrs().  DAMON can automatically adjust the perf event sampling
> > +frequency to minimize old report drops due to the buffer size limit.
> > +
> > +The user further sets a DAMON probe.  The probe hits for a given DAMON region
> > +if a store instruction for a memory in the region that was executed from a
> > +given CPUs group exists in the report buffer.  As a result, the probe hit
> > +counts represent relative frequency of writes from a given CPUs group to a
> > +given memory region.
> 
> [Severity: Medium]
> In the Data Attributes Reporting section of Documentation/mm/damon/design.rst,
> how can the DAMON probe determine whether a sampled store instruction targeted
> a given DAMON memory region if the callback only saves the CPU instruction and
> CPU ID from a generic CPU cycles counter?
> 
> A sampled instruction (or instruction pointer) only encodes the opcode and
> register operands, not the effective virtual or physical data address being
> written. Saving only the instruction and executing CPU ID via
> damon_report_attrs() does not appear to provide the target memory address
> needed to match against a DAMON region when the probe inspects the buffer.
> 
> Even if the perf overflow callback attempted to resolve the target data
> address at NMI time, generic CPU cycle counter interrupts
> (PERF_COUNT_HW_CPU_CYCLES) are non-precise on out-of-order CPUs and suffer
> from skid. At NMI time, regs->ip typically points to a subsequent instruction
> rather than the instruction that caused the overflow, and the register values
> in struct pt_regs may already have been modified.
> 
> Furthermore, without hardware data-address sampling features (such as Intel
> PEBS, AMD IBS, or Arm SPE populating perf_sample_data->addr or phys_addr,
> which are scoped for Milestone 3), resolving the effective address from a raw
> instruction would require fetching instruction bytes in NMI context, running
> an architecture-specific instruction decoder and address-calculation emulator,
> and translating virtual addresses to physical addresses for paddr monitoring.
> 
> How is the target memory address intended to be obtained and matched to DAMON
> regions in Milestone 2 with generic CPU cycle sampling?

Good question.  damon_report_attr() will save the register contents and pid
into report, too.  And the probe in kdamond context will decode the instruction
and get the memory address.  For physical address, it will also do the page
table walk of the process of the pid.

> 
> -- 
> Sashiko AI review · https://sashiko.dev/#/patchset/20261017195747.89961-1-sj@kernel.org?part=1


Thanks,
SJ

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[Ravi Jonnalagadda]

Hi SJ,

On Sun, 18 Oct 2026 13:08:07 -0700 SJ Park <sj@kernel.org> wrote:

> Good question.  damon_report_attr() will save the register contents and pid
> into report, too.  And the probe in kdamond context will decode the instruction
> and get the memory address.  For physical address, it will also do the page
> table walk of the process of the pid.

I measured what the samples carry, to see how far decode would go.

On AMD, cpu-cycles:p is IBS Op, and its memory-op samples carry a
hardware data address.  On Intel (Granite Rapids), cycles samples carry
no data address at all; the PEBS memory events
(MEM_INST_RETIRED.ALL_LOADS/ALL_STORES) do.

Share of samples with an address inside the 2 GiB test buffer (random
loads and stores; perf record -d; one run per event):

  Xeon 6960P   cycles:p / cycles:pp / :ppp     0 %
  Xeon 6960P   mem_inst_retired.*:pp          100 %
  EPYC 9555P   cycles:p, cycles:pp, ibs_op//  ~21 % of all samples
                                              (the memory-op ones)

How often the saved AX holds the target address when AX is the base
register (perf record -d --intr-regs=AX; store to (%rax) vs
pointer-chase load mov (%rax), %rax):

  Xeon 6960P   store loop 100 %    pointer-chase loop 0 %
  EPYC 9555P   store loop   0 %    pointer-chase loop 0 %

PEBS saves registers just after the instruction retires, so a load that
overwrites its base register defeats decode.  IBS saves no general
registers (perf reports the registers at the NMI), so decode is not
reliable on AMD.  Decode in kdamond must also survive exec, munmap, exit
and pid reuse.  These are synthetic loops chosen to separate the cases;
how often real code hits the failing case is not measured.

For the physical address, taking it at sample time, while the task is
current, avoids walking a page table that may have changed by the time
kdamond runs.  IBS supplies it when valid; on Intel the producer can
call perf_prepare_sample() in the overflow NMI to do the walk.

So the suggestion is for the first report type to use the PMU-captured
address and load/store flag (AMD cpu-cycles:p; on Intel, the PEBS
memory events, which moves them into milestone 2).  Your call, of
course.

Thanks,
Ravi

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[Ravi Jonnalagadda]

Hi SJ,

Thank you for sharing the design early.  Short version: the design makes
sense to me, with one main suggestion.  The first report type should use
the data address the PMU captures at sample time, rather than decode the
instruction later in kdamond.  The measurements behind it are in my
reply to your answer to sashiko.

On Sat, 17 Oct 2026 12:57:45 -0700 SJ Park <sj@kernel.org> wrote:

> Does the idea for perf CPU cycle sampling based access monitoring makes
> sense?

On AMD, yes, with precise_ip (cpu-cycles:p routes to IBS Op): its
memory-op samples carry a data address and load/store type.  On Intel,
even precise cycles samples (cycles:pp) carry no data address; PEBS
memory events do.  Decode from saved registers worked for one of two
test loops on Intel and for neither on AMD (details in the other reply),
hence the suggestion above.

> Does the design provides clear idea about how it can be extended for the
> future extensions that already scoped for milestone 3 (AMD IBS, Intel
> PEBS, Arm SPE)?

Yes, if the report has an address field with a validity bit and carries
the producing report-type (attr) id.  Then IBS, PEBS, Arm SPE (through
an adapter, since SPE uses the perf AUX buffer) and device-side
monitors all produce the same report, into the same queues and probes.

> For the foreseeable extensions (AMD IBS, Intel PEBS, Arm SPE), would the
> required change minimum and maintainable?

Yes, if staged.  The sample-filter matcher and its sysfs already exist
per context; attaching them per probe is the main change.  For example:

  M2.1  in-kernel only: damon_report_attrs(), the queue, region
        lookup, the perf producer; kunit, no new sysfs
  M2.2  your per-CPU write-only example, set up from user space:
        attr_reports files, per-filter report_queue,
        write/cpumask/threads probe filters
  M2.3  counters, a silence check (tracepoint/debugfs first), tuning
  M3    more producers: Arm SPE, device monitors

Implementation notes:

1. One sampler per CPU per attr, owned by attr_reports (as in your
   damon/next usage doc) and shared by all contexts.  IBS has one unit
   per CPU, so perf time-slices multiple events: on EPYC 9555P one
   ibs_op event got 67,140 samples; two on the same CPU got
   33,547 + 33,535.  On Intel the event also needs wakeup_events = 1,
   or an in-kernel handler sees only the last record of each
   large-PEBS batch.

2. A lockless queue, as your NOTE in damon_report_access() anticipates.
   Today's buffer is documented "May sleep" and uses mutex_trylock(),
   which interrupt context cannot use; its inclusive jiffy staleness
   check can also count one report in up to three intervals (HZ=250,
   5 ms interval).  A simple design: a per-CPU circular array with a
   write counter and a per-slot seqcount_t (using
   raw_write_seqcount_begin/end() for NMI safety), overwriting the
   oldest slot.  Readers never consume; each context keeps its
   last-seen write counter per CPU.  A nested writer (e.g. an NMI over
   a task-context write) drops its report and counts it.

3. Report fields: paddr, vaddr, size, time, cpu, tid, tgid, the
   producer's attr id, and flags (is_write, plus a validity bit per
   address, since 0 is a valid paddr).  Store global ids; translate
   namespaces at the filter.

4. Binary search over a sorted region snapshot (your todo in
   kdamond_apply_access_report()) beats a linear walk from about 2
   reports per interval for 1,000 regions (a model, to be measured).

5. Samplers can stop silently: in a report on the v4 thread [1], IBS
   Op samples stopped after about 10 minutes on a Ryzen 5 5600H, but
   not with the NMI watchdog enabled.  So count dropped reports and
   check for silence against an independent reference.  For tuning,
   step sample_period rather than use frequency mode, and do not raise
   perf_event_max_sample_rate: on an early DAMON PEBS prototype (Xeon
   6960P), perf cut it from 100000 to 10000 in about 20 seconds.

Three decisions would unblock M2.1:

  - The first report type: the PMU-captured address (suggested), or
    decode from saved registers in kdamond (your current plan).
  - Whether the report carries the producer's attr id (suggested: yes).
  - The queue: the per-CPU seqcount design in note 2 (suggested), or
    ftrace's ring buffer with one iterator per reader.

One question on the ABI: the usage document in your damon/next tree adds
a report_queue file under each probe filter.  Is it meant as "this
filter tests reports in queue N"?

Happy to write, review or test any piece: the queue with kunit, the
AMD/Intel perf producer, a decode-accuracy study on real workloads, or
runs on Xeon 6960P and EPYC 9555P.  Several exist in RFC v4 [2].

Thanks,
Ravi

[1] https://lore.kernel.org/r/20261017211613.16456-2-aethernet65535@gmail.com
[2] https://lore.kernel.org/r/20261005-damon-perf-rfc-v3-send-2026-10-03-v4-0-b03452e137f3@gmail.com

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

Hi Ravi,

On Wed, 21 Oct 2026 21:17:24 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:

> Hi SJ,
> 
> Thank you for sharing the design early.  Short version: the design makes
> sense to me, with one main suggestion.  The first report type should use
> the data address the PMU captures at sample time, rather than decode the
> instruction later in kdamond.  The measurements behind it are in my
> reply to your answer to sashiko.

Thank you for your detailed review and testing of the first design!  I'd still
like to try decoding idea first, mainly for quick prototyping.

> 
> On Sat, 17 Oct 2026 12:57:45 -0700 SJ Park <sj@kernel.org> wrote:
> 
> > Does the idea for perf CPU cycle sampling based access monitoring makes
> > sense?
> 
> On AMD, yes, with precise_ip (cpu-cycles:p routes to IBS Op): its
> memory-op samples carry a data address and load/store type.  On Intel,
> even precise cycles samples (cycles:pp) carry no data address; PEBS
> memory events do.  Decode from saved registers worked for one of two
> test loops on Intel and for neither on AMD (details in the other reply),
> hence the suggestion above.

My current understanding of perf event is not good enough to deeply understand
those.  I will start with full registers capture-decode option and revisit
this.

> 
> > Does the design provides clear idea about how it can be extended for the
> > future extensions that already scoped for milestone 3 (AMD IBS, Intel
> > PEBS, Arm SPE)?
> 
> Yes, if the report has an address field with a validity bit and carries
> the producing report-type (attr) id.  Then IBS, PEBS, Arm SPE (through
> an adapter, since SPE uses the perf AUX buffer) and device-side
> monitors all produce the same report, into the same queues and probes.

Thank you for confirming.  And yes, I think we can add more (union) fields to
the report structure.

However, could you further clarify what are the validity bit and attr id, and
why those are needed?  I'm still newbie to perf event, please bear in mind with
me.

> 
> > For the foreseeable extensions (AMD IBS, Intel PEBS, Arm SPE), would the
> > required change minimum and maintainable?
> 
> Yes, if staged.  The sample-filter matcher and its sysfs already exist
> per context; attaching them per probe is the main change.  For example:
> 
>   M2.1  in-kernel only: damon_report_attrs(), the queue, region
>         lookup, the perf producer; kunit, no new sysfs
>   M2.2  your per-CPU write-only example, set up from user space:
>         attr_reports files, per-filter report_queue,
>         write/cpumask/threads probe filters
>   M2.3  counters, a silence check (tracepoint/debugfs first), tuning
>   M3    more producers: Arm SPE, device monitors
> 
> Implementation notes:
> 
> 1. One sampler per CPU per attr, owned by attr_reports (as in your
>    damon/next usage doc)

The things in damon/next are just my sketches that are not yet ready for
review.  It will nearly all be re-written.  I'd recommend to ignore those for
now.

>    and shared by all contexts.  IBS has one unit
>    per CPU, so perf time-slices multiple events: on EPYC 9555P one
>    ibs_op event got 67,140 samples; two on the same CPU got
>    33,547 + 33,535.  On Intel the event also needs wakeup_events = 1,
>    or an in-kernel handler sees only the last record of each
>    large-PEBS batch.
> 
> 2. A lockless queue, as your NOTE in damon_report_access() anticipates.
>    Today's buffer is documented "May sleep" and uses mutex_trylock(),
>    which interrupt context cannot use; its inclusive jiffy staleness
>    check can also count one report in up to three intervals (HZ=250,
>    5 ms interval).  A simple design: a per-CPU circular array with a
>    write counter and a per-slot seqcount_t (using
>    raw_write_seqcount_begin/end() for NMI safety), overwriting the
>    oldest slot.  Readers never consume; each context keeps its
>    last-seen write counter per CPU.  A nested writer (e.g. an NMI over
>    a task-context write) drops its report and counts it.

I agree the queue shouldn't sleep.  You can ignore those in damon/next.
Nonetheless, I'd like to start from global queue rather than per-CPU queue.  If
the performance is concern, we can later consider optimization.  Let me know if
I'm missing something.

> 
> 3. Report fields: paddr, vaddr, size, time, cpu, tid, tgid, the
>    producer's attr id, and flags (is_write, plus a validity bit per
>    address, since 0 is a valid paddr).  Store global ids; translate
>    namespaces at the filter.

Yes, this kind of information will be added.  Some of those could be union.
And we could extend the fields as needed.

> 
> 4. Binary search over a sorted region snapshot (your todo in
>    kdamond_apply_access_report()) beats a linear walk from about 2
>    reports per interval for 1,000 regions (a model, to be measured).

Yes, this efficient region-report matching would be important.  We could also
consider starting with linear walk and later optimize, though.

> 
> 5. Samplers can stop silently: in a report on the v4 thread [1], IBS
>    Op samples stopped after about 10 minutes on a Ryzen 5 5600H, but
>    not with the NMI watchdog enabled.  So count dropped reports and
>    check for silence against an independent reference.  For tuning,
>    step sample_period rather than use frequency mode, and do not raise
>    perf_event_max_sample_rate: on an early DAMON PEBS prototype (Xeon
>    6960P), perf cut it from 100000 to 10000 in about 20 seconds.

Unfortunately my understanding of perf event is not yet good enough to
understand this in depth.  I will revisit this after starting implementation.

> 
> Three decisions would unblock M2.1:
> 
>   - The first report type: the PMU-captured address (suggested), or
>     decode from saved registers in kdamond (your current plan).

I'd like to start from decoding option.  It will be easier to test on my setup
and therefore I will be able to share first prototype earlier.  Also, there are
users who cannot use address-capturing PMUs.  For example, some users need to
run DAMON inside virtual machines that might not support virtual PMUs.

>   - Whether the report carries the producer's attr id (suggested: yes).

We can add any field if needed.  I'm not very sure what attr id is and why
needed, though, as I asked above.

>   - The queue: the per-CPU seqcount design in note 2 (suggested), or
>     ftrace's ring buffer with one iterator per reader.

My current plan is to use DAMON's original simple lockless and global queue per
report queue.  Let me know if the report queue design is not clearly explaining
what it would look like, or there is a clear problem.

> 
> One question on the ABI: the usage document in your damon/next tree adds
> a report_queue file under each probe filter.  Is it meant as "this
> filter tests reports in queue N"?

Please simply ignore those in damon/next for now.  Those are not meant to be
reviewed.  I am still redesigning the interface.  The high level design in the
initial mail of this thread is still unchanged, though.  Feel free to ask
questions if any of the design is unclear or concerning.

> 
> Happy to write, review or test any piece: the queue with kunit, the
> AMD/Intel perf producer, a decode-accuracy study on real workloads, or
> runs on Xeon 6960P and EPYC 9555P.  Several exist in RFC v4 [2].

As soon as this design discussion is completed, I will start prototype
implementation and share it as the next RFC version, as soon as possible.  Your
review and test of the version will be super helpful.  I may also ask you some
questions for perf event and your prototypes.  Thanks in advance for your help.

There are competing works, though.  It would take time.  I expect about one
month at minimum.  It might take longer; Thanksgiving holidays could be helpful
to give me a time to concentrate.  Please bear in mind.  But once the initial
prototype is implemented, I believe we could quickly iterate and meet the ETA
(LSFMMBPF'27).

> 
> Thanks,
> Ravi
> 
> [1] https://lore.kernel.org/r/20261017211613.16456-2-aethernet65535@gmail.com
> [2] https://lore.kernel.org/r/20261005-damon-perf-rfc-v3-send-2026-10-03-v4-0-b03452e137f3@gmail.com


Thanks,
SJ

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

On Wed, 21 Oct 2026 21:17:08 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:

> Hi SJ,
> 
> On Sun, 18 Oct 2026 13:08:07 -0700 SJ Park <sj@kernel.org> wrote:
> 
> > Good question.  damon_report_attr() will save the register contents and pid
> > into report, too.  And the probe in kdamond context will decode the instruction
> > and get the memory address.  For physical address, it will also do the page
> > table walk of the process of the pid.
> 
> I measured what the samples carry, to see how far decode would go.
> 
> On AMD, cpu-cycles:p is IBS Op, and its memory-op samples carry a
> hardware data address.  On Intel (Granite Rapids), cycles samples carry
> no data address at all; the PEBS memory events
> (MEM_INST_RETIRED.ALL_LOADS/ALL_STORES) do.
> 
> Share of samples with an address inside the 2 GiB test buffer (random
> loads and stores; perf record -d; one run per event):
> 
>   Xeon 6960P   cycles:p / cycles:pp / :ppp     0 %
>   Xeon 6960P   mem_inst_retired.*:pp          100 %
>   EPYC 9555P   cycles:p, cycles:pp, ibs_op//  ~21 % of all samples
>                                               (the memory-op ones)
> 
> How often the saved AX holds the target address when AX is the base
> register (perf record -d --intr-regs=AX; store to (%rax) vs
> pointer-chase load mov (%rax), %rax):
> 
>   Xeon 6960P   store loop 100 %    pointer-chase loop 0 %
>   EPYC 9555P   store loop   0 %    pointer-chase loop 0 %
> 
> PEBS saves registers just after the instruction retires, so a load that
> overwrites its base register defeats decode.  IBS saves no general
> registers (perf reports the registers at the NMI), so decode is not
> reliable on AMD.  Decode in kdamond must also survive exec, munmap, exit
> and pid reuse.  These are synthetic loops chosen to separate the cases;
> how often real code hits the failing case is not measured.
> 
> For the physical address, taking it at sample time, while the task is
> current, avoids walking a page table that may have changed by the time
> kdamond runs.  IBS supplies it when valid; on Intel the producer can
> call perf_prepare_sample() in the overflow NMI to do the walk.

Thank you for these testing, Ravi.  Unfortunately I'm still learning perf event
and therefore not yet fully understanding these tests.  I will revisit after
gaining more understanding.

> 
> So the suggestion is for the first report type to use the PMU-captured
> address and load/store flag (AMD cpu-cycles:p; on Intel, the PEBS
> memory events, which moves them into milestone 2).  Your call, of
> course.

Meanwhile, I will start with the current plan (capture register contents in NMI
and decode in kdamond) if you don't have strong concern.


Thanks,
SJ

[...]

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[Ravi Jonnalagadda]

Hi SJ,

On Wed, 21 Oct 2026 22:32:26 -0700 SJ Park <sj@kernel.org> wrote:

> On Wed, 21 Oct 2026 21:17:08 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:
>
[...]
> > PEBS saves registers just after the instruction retires, so a load that
> > overwrites its base register defeats decode.  IBS saves no general
> > registers (perf reports the registers at the NMI), so decode is not
> > reliable on AMD.  Decode in kdamond must also survive exec, munmap, exit
> > and pid reuse.  These are synthetic loops chosen to separate the cases;
> > how often real code hits the failing case is not measured.
[...]
>
> Thank you for these testing, Ravi.  Unfortunately I'm still learning perf event
> and therefore not yet fully understanding these tests.  I will revisit after
> gaining more understanding.
>
> > So the suggestion is for the first report type to use the PMU-captured
> > address and load/store flag (AMD cpu-cycles:p; on Intel, the PEBS
> > memory events, which moves them into milestone 2).  Your call, of
> > course.
>
> Meanwhile, I will start with the current plan (capture register contents in NMI
> and decode in kdamond) if you don't have strong concern.

No strong concern.  Starting with decode makes sense, and VMs are a
good reason for it: in the KVM setups tried on Intel and AMD hosts, no
guest got a data address from the PMU, whereas every guest had a
software CPU clock (PERF_COUNT_SW_CPU_CLOCK) or basic hardware cycle
counter (PERF_COUNT_HW_CPU_CYCLES).

As quick background on why those earlier test tables brought up PEBS
and IBS: in perf_event, sampling CPU cycles has two modes, selected by
attr.precise_ip (the :p suffix in perf record):

  - Plain cycle or timer sampling (attr.precise_ip = 0, the default):
    a counter overflows and raises a normal interrupt/NMI.  The
    hardware does not record a data address; the handler just sees
    pt_regs where the CPU stopped between instructions.  Even though
    the interrupt lands a few instructions after the overflow ("skid"),
    regs->ip is the next instruction about to run and pt_regs holds
    its untouched input registers -- so decoding regs->ip gives the
    address it is about to access.

  - Hardware "precise" sampling (attr.precise_ip > 0, which routes to
    Intel PEBS or AMD IBS): the PMU hardware itself snapshots the
    instruction and, on supported events, the data address it touched.
    My earlier tables tested this mode.  Because PEBS records registers
    after the instruction retires (when a load into its own base
    register has already overwritten it) and IBS does not capture
    general registers, software decode breaks there -- which is fine,
    since in that mode you use the hardware-captured address instead
    (your Milestone 3).

So there is no need to revisit those earlier test tables for your
Milestone 2 prototype.

Two small tips for the prototype:

  - use PERF_COUNT_HW_CPU_CYCLES (or PERF_COUNT_SW_CPU_CLOCK in VMs
    without a virtual PMU) with the default attr.precise_ip = 0;
  - skip, and count, samples whose instruction does not access memory;
    in typical programs, only ~15-60% of cycle samples land on a load
    or store.

If you want to inspect what those samples look like from user space on
a program:

  R=AX,BX,CX,DX,SI,DI,BP,SP,IP,R8,R9,R10,R11,R12,R13,R14,R15
  perf record -c 2000003 -e cycles:u --intr-regs=$R -- <program>
  perf script -F ip,iregs,insn

(In a guest without hardware cycle counters, use -e cpu-clock:u
instead.)  Each sample shows the IP, the registers and the instruction
bytes to decode.

Thanks,
Ravi

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[Ravi Jonnalagadda]

Hi SJ,

On Wed, 21 Oct 2026 22:29:12 -0700 SJ Park <sj@kernel.org> wrote:

> Hi Ravi,
>
> On Wed, 21 Oct 2026 21:17:24 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:
>
[...]
>
> Thank you for your detailed review and testing of the first design!  I'd still
> like to try decoding idea first, mainly for quick prototyping.

Sounds good.  Starting with decode makes sense, especially for VMs
where hardware address-sampling PMUs are often unavailable; some tips
for the prototype are in the other reply [1].

[...]
> > Yes, if the report has an address field with a validity bit and carries
> > the producing report-type (attr) id.  Then IBS, PEBS, Arm SPE (through
> > an adapter, since SPE uses the perf AUX buffer) and device-side
> > monitors all produce the same report, into the same queues and probes.
>
> Thank you for confirming.  And yes, I think we can add more (union) fields to
> the report structure.
>
> However, could you further clarify what are the validity bit and attr id, and
> why those are needed?  I'm still newbie to perf event, please bear in mind with
> me.

Neither is perf-specific:

- Validity bit (or union tag): a way to tell whether a report carries a
  virtual address, a physical address, or both.  For example, a decoded
  sample has no physical address if the page is not mapped.  If you use
  a union with a type field, that already covers it.

- Attr id: an id of the producer (report type) that made the report.  If
  several producers later feed one queue (e.g. perf sampling and a
  device-side monitor), a probe can filter on, or weigh, reports by
  their source.  If each queue only ever has one producer, the queue
  already identifies it, and the field is not needed.

[...]
> > 4. Binary search over a sorted region snapshot (your todo in
> >    kdamond_apply_access_report()) beats a linear walk from about 2
> >    reports per interval for 1,000 regions (a model, to be measured).
>
> Yes, this efficient region-report matching would be important.  We could also
> consider starting with linear walk and later optimize, though.

Agreed; starting with a linear walk for the prototype is fine, and once
it is out, happy to measure where binary search starts to matter.

[...]
> > 5. Samplers can stop silently: in a report on the v4 thread [1], IBS
[...]
> Unfortunately my understanding of perf event is not yet good enough to
> understand this in depth.  I will revisit this after starting implementation.

Sure.  The one part worth keeping in mind from the start: a sampler can
stop without any error, so a counter of reports received and dropped
helps tell "no accesses" from "no samples".

[...]
> >   - The queue: the per-CPU seqcount design in note 2 (suggested), or
> >     ftrace's ring buffer with one iterator per reader.
>
> My current plan is to use DAMON's original simple lockless and global queue per
> report queue.  Let me know if the report queue design is not clearly explaining
> what it would look like, or there is a clear problem.

A global queue is fine to start with, and the design is clear.  One
thing to keep in mind: perf writers run in interrupt or NMI context on
all CPUs at once, and an NMI can arrive in the middle of another write.
So reserving a slot needs to be lock-free, and a reader should not see
a half-written report.  If the design already covers that, there is
nothing to add.

[...]
> As soon as this design discussion is completed, I will start prototype
> implementation and share it as the next RFC version, as soon as possible.  Your
> review and test of the version will be super helpful.  I may also ask you some
> questions for perf event and your prototypes.  Thanks in advance for your help.
>
> There are competing works, though.  It would take time.  I expect about one
> month at minimum.  It might take longer; Thanksgiving holidays could be helpful
> to give me a time to concentrate.  Please bear in mind.  But once the initial
> prototype is implemented, I believe we could quickly iterate and meet the ETA
> (LSFMMBPF'27).

No rush on the timeline; that plan sounds good.  Please feel free to
ask any time, and I will be happy to share the test programs and review
and test the prototype when it is out.

Thanks,
Ravi

[1] https://lore.kernel.org/r/20261023194848.2449-1-ravis.opensrc@gmail.com

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

On Fri, 23 Oct 2026 12:51:22 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:

> Hi SJ,
> 
> On Wed, 21 Oct 2026 22:29:12 -0700 SJ Park <sj@kernel.org> wrote:
> 
> > Hi Ravi,
> >
> > On Wed, 21 Oct 2026 21:17:24 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:
> >
> [...]
> >
> > Thank you for your detailed review and testing of the first design!  I'd still
> > like to try decoding idea first, mainly for quick prototyping.
> 
> Sounds good.  Starting with decode makes sense, especially for VMs
> where hardware address-sampling PMUs are often unavailable; some tips
> for the prototype are in the other reply [1].

Thank you for confirming and the tips!

> 
> [...]
> > > Yes, if the report has an address field with a validity bit and carries
> > > the producing report-type (attr) id.  Then IBS, PEBS, Arm SPE (through
> > > an adapter, since SPE uses the perf AUX buffer) and device-side
> > > monitors all produce the same report, into the same queues and probes.
> >
> > Thank you for confirming.  And yes, I think we can add more (union) fields to
> > the report structure.
> >
> > However, could you further clarify what are the validity bit and attr id, and
> > why those are needed?  I'm still newbie to perf event, please bear in mind with
> > me.
> 
> Neither is perf-specific:
> 
> - Validity bit (or union tag): a way to tell whether a report carries a
>   virtual address, a physical address, or both.  For example, a decoded
>   sample has no physical address if the page is not mapped.  If you use
>   a union with a type field, that already covers it.

Thank you for clarifying.  Yes, it may have a union with a type field.

> 
> - Attr id: an id of the producer (report type) that made the report.  If
>   several producers later feed one queue (e.g. perf sampling and a
>   device-side monitor), a probe can filter on, or weigh, reports by
>   their source.  If each queue only ever has one producer, the queue
>   already identifies it, and the field is not needed.

Thank you for clarifying.  I agree this kind of information will be needed, and
I will make sure it is in the report struct.

> 
> [...]
> > > 4. Binary search over a sorted region snapshot (your todo in
> > >    kdamond_apply_access_report()) beats a linear walk from about 2
> > >    reports per interval for 1,000 regions (a model, to be measured).
> >
> > Yes, this efficient region-report matching would be important.  We could also
> > consider starting with linear walk and later optimize, though.
> 
> Agreed; starting with a linear walk for the prototype is fine, and once
> it is out, happy to measure where binary search starts to matter.

Cool!

> 
> [...]
> > > 5. Samplers can stop silently: in a report on the v4 thread [1], IBS
> [...]
> > Unfortunately my understanding of perf event is not yet good enough to
> > understand this in depth.  I will revisit this after starting implementation.
> 
> Sure.  The one part worth keeping in mind from the start: a sampler can
> stop without any error, so a counter of reports received and dropped
> helps tell "no accesses" from "no samples".

Good point.  I will try to make it at least easy to debug.

> 
> [...]
> > >   - The queue: the per-CPU seqcount design in note 2 (suggested), or
> > >     ftrace's ring buffer with one iterator per reader.
> >
> > My current plan is to use DAMON's original simple lockless and global queue per
> > report queue.  Let me know if the report queue design is not clearly explaining
> > what it would look like, or there is a clear problem.
> 
> A global queue is fine to start with, and the design is clear.  One
> thing to keep in mind: perf writers run in interrupt or NMI context on
> all CPUs at once, and an NMI can arrive in the middle of another write.
> So reserving a slot needs to be lock-free, and a reader should not see
> a half-written report.  If the design already covers that, there is
> nothing to add.

Design is still in progress, but definitely this should be covered.  Thank you
for calling out!

> 
> [...]
> > As soon as this design discussion is completed, I will start prototype
> > implementation and share it as the next RFC version, as soon as possible.  Your
> > review and test of the version will be super helpful.  I may also ask you some
> > questions for perf event and your prototypes.  Thanks in advance for your help.
> >
> > There are competing works, though.  It would take time.  I expect about one
> > month at minimum.  It might take longer; Thanksgiving holidays could be helpful
> > to give me a time to concentrate.  Please bear in mind.  But once the initial
> > prototype is implemented, I believe we could quickly iterate and meet the ETA
> > (LSFMMBPF'27).
> 
> No rush on the timeline; that plan sounds good.  Please feel free to
> ask any time, and I will be happy to share the test programs and review
> and test the prototype when it is out.

Appreciate your continued support, that will be super helpful!

> 
> Thanks,
> Ravi
> 
> [1] https://lore.kernel.org/r/20261023194848.2449-1-ravis.opensrc@gmail.com


Thanks,
SJ

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

On Fri, 23 Oct 2026 12:48:48 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:

> Hi SJ,
> 
> On Wed, 21 Oct 2026 22:32:26 -0700 SJ Park <sj@kernel.org> wrote:
> 
> > On Wed, 21 Oct 2026 21:17:08 -0700 Ravi Jonnalagadda <ravis.opensrc@gmail.com> wrote:
> >
> [...]
> > > PEBS saves registers just after the instruction retires, so a load that
> > > overwrites its base register defeats decode.  IBS saves no general
> > > registers (perf reports the registers at the NMI), so decode is not
> > > reliable on AMD.  Decode in kdamond must also survive exec, munmap, exit
> > > and pid reuse.  These are synthetic loops chosen to separate the cases;
> > > how often real code hits the failing case is not measured.
> [...]
> >
> > Thank you for these testing, Ravi.  Unfortunately I'm still learning perf event
> > and therefore not yet fully understanding these tests.  I will revisit after
> > gaining more understanding.
> >
> > > So the suggestion is for the first report type to use the PMU-captured
> > > address and load/store flag (AMD cpu-cycles:p; on Intel, the PEBS
> > > memory events, which moves them into milestone 2).  Your call, of
> > > course.
> >
> > Meanwhile, I will start with the current plan (capture register contents in NMI
> > and decode in kdamond) if you don't have strong concern.
> 
> No strong concern.  Starting with decode makes sense, and VMs are a
> good reason for it: in the KVM setups tried on Intel and AMD hosts, no
> guest got a data address from the PMU, whereas every guest had a
> software CPU clock (PERF_COUNT_SW_CPU_CLOCK) or basic hardware cycle
> counter (PERF_COUNT_HW_CPU_CYCLES).
> 
> As quick background on why those earlier test tables brought up PEBS
> and IBS: in perf_event, sampling CPU cycles has two modes, selected by
> attr.precise_ip (the :p suffix in perf record):
> 
>   - Plain cycle or timer sampling (attr.precise_ip = 0, the default):
>     a counter overflows and raises a normal interrupt/NMI.  The
>     hardware does not record a data address; the handler just sees
>     pt_regs where the CPU stopped between instructions.  Even though
>     the interrupt lands a few instructions after the overflow ("skid"),
>     regs->ip is the next instruction about to run and pt_regs holds
>     its untouched input registers -- so decoding regs->ip gives the
>     address it is about to access.
> 
>   - Hardware "precise" sampling (attr.precise_ip > 0, which routes to
>     Intel PEBS or AMD IBS): the PMU hardware itself snapshots the
>     instruction and, on supported events, the data address it touched.
>     My earlier tables tested this mode.  Because PEBS records registers
>     after the instruction retires (when a load into its own base
>     register has already overwritten it) and IBS does not capture
>     general registers, software decode breaks there -- which is fine,
>     since in that mode you use the hardware-captured address instead
>     (your Milestone 3).
> 
> So there is no need to revisit those earlier test tables for your
> Milestone 2 prototype.
> 
> Two small tips for the prototype:
> 
>   - use PERF_COUNT_HW_CPU_CYCLES (or PERF_COUNT_SW_CPU_CLOCK in VMs
>     without a virtual PMU) with the default attr.precise_ip = 0;
>   - skip, and count, samples whose instruction does not access memory;
>     in typical programs, only ~15-60% of cycle samples land on a load
>     or store.
> 
> If you want to inspect what those samples look like from user space on
> a program:
> 
>   R=AX,BX,CX,DX,SI,DI,BP,SP,IP,R8,R9,R10,R11,R12,R13,R14,R15
>   perf record -c 2000003 -e cycles:u --intr-regs=$R -- <program>
>   perf script -F ip,iregs,insn
> 
> (In a guest without hardware cycle counters, use -e cpu-clock:u
> instead.)  Each sample shows the IP, the registers and the instruction
> bytes to decode.

Thank you for these useful information an tips!


Thanks,
SJ