[Linux Memory Hotness and Promotion] Notes from August 14, 2025

[Linux Memory Hotness and Promotion] Notes from August 14, 2025

[David Rientjes]

Hi everybody,

Here are the notes from the last Linux Memory Hotness and Promotion call
that happened on Thursday, August 14.  Thanks to everybody who was 
involved!

These notes are intended to bring people up to speed who could not attend 
the call as well as keep the conversation going in between meetings.

----->o-----
I asked about the current status of the single source of truth for page
hotness and then kmigrated.  Bharata noted that he had recently sent the
latest revision of the series[1], which includes a lot of cleanups and
integrates both IBS driver and klruscand as sources of hotness
information.

The single source of truth for page hotness information was also renamed,
but kmigrated was then renamed to kpromoted (it was only responsible for
promotion of memory from lower tiers).

Raghu also that he posted the latest revision of kscand[2] and was
interested in discussing after another round of code reviews.  We
discussed integration of the slowtier page promotion based on PTE A bit
series and its integration into the single source of hotness information.
It's been experimented with, but the integration hasn't been done yet
until later alignment.

Raghu clarified that his series could be standalone and integrated later
when needed.  The scanning and migration part are completely decoupled;
consuming the hotness information was still a work in progress and could
change over time.

----->o-----
Kinsey Ho updated on the status of klruscand.  As this was a proof of
concept, the next step was to clean up the code and then try it internally
before proposing it for integration upstream.  Wei Xu noted that this was
a proof of concept to show that MGLRU could be used for PTE scanning for
promotion purposes.

Kinsey updated that he planned on sharing the latest series upstream by
mid September.  AMD, as using this as a reference implementation, could
continue to integrate this for an RFC patch series as an additional proof
of concept.

----->o-----
Bharata noted that during scanning of memory with Raghu's series, a good
estimate of the target node id could be derived which was very valuable.
All of the ratelimiting support should likely be part of Bharata's series
as the one controlling promotion.

Raghu agreed that the migration part should be combined betwee the two
patch series.  Raghu noted his series included per-mm throttling that
could be integrated.  To summarize, Bharata noted that the scanning part
and optimizations for target node id detection from Raghu's patches would
drive the common infrastructure that Bharata was developing for
migration (to include things like ratelimiting).

Bharata asked Kinsey how the target node id would be derived from
klruscand.  Kinsey noted that this was still an open question.

Jonathan Cameron asked why don't we just use a page fault hint for the
super hot pages that we want to promote.  For example, once we've decided
that we want to promote pages, just invalidate the ptes and wait for a
page fault (since they are hot, this will happen very soon).  Wei noted
there would be pros and cons to this and one of the cons would be that we
need to take a page fault.

----->o-----
SeongJae Park noted similarities in the DAMON design which supports
multiple sources of page hotness information (the low level access
collection and a middle layer that would control over the overhead).  He
is currently working on hint faults: the access bit does not include the
cpu information that the memory is being accessed from.  He's using NUMA
hint faults for this and discovered that it is very effective.  He is
trying to abstract this better in the Linux kernel where it can be used
for multiple use cases including DAMON, NUMA Balancing, and kpromoted.

SJ asked about the timelines for leveraging NUMA hint faults as part of
kpromoted.  Bharata referred to his kmigrated patch series where there
was a proof of concept for separating out the migration part in batched
and async modes for NUMA Balancing.  He experimented with this for
kpromoted but it's not yet posted.  His goal is to leverage NUMAB=2 to
feed into kmigrated.

----->o-----
Next meeting will be on Thursday, August 28 at 8:30am PDT (UTC-7),
everybody is welcome: https://meet.google.com/jak-ytdx-hnm

Topics for the next meeting:

 - update on Bharata's subsystem for hot page detection and promotion
   after latest series of changes were posted, including promotion
   ratelimiting
 - discuss Raghu's slowtier page promotion based on PTE A bit approach
   and any timelines for integration into the above to hand-off the
   promotion work
 - update from Bharata on integration of NUMA Balancing memory tiering
   support into kmigrated
 - update on status of sharing the cleanup of klruscand in mid September
 - new topic: discuss leveraging non-temporal stores to migrate memory to
   save on memory bandwidth
 - enlightening migrate_pages() for hardware assists and how this work
   will be charged to userspace
 - update on sharing Google approach for both to overlap the shared goals
   and converge where possible
 - discuss proactive demotion interface as an extension to memory.reclaim
 - discuss overall testing and benchmarking methodology for various
   approaches as we go along
   + minimal viable infrastructure, testing workloads, and metrics of
     interest to collect

Please let me know if you'd like to propose additional topics for
discussion, thank you!

[1] https://marc.info/?l=linux-kernel&m=175517926006575&w=2
[2]
https://lore.kernel.org/linux-mm/20250814153307.1553061-1-raghavendra.kt@amd.com/T/#t