[Linux Memory Hotness and Promotion] Notes from October 22, 2026

[Linux Memory Hotness and Promotion] Notes from October 22, 2026

[David Rientjes]

Hi everybody,

Here are the notes from the last Linux Memory Hotness and Promotion call
that happened on Thursday, October 22.  Thanks to everybody who was 
involved!

These notes are intended to bring people up to speed who could not attend 
the call as well as keep the conversation going in between meetings.

----->o-----
We debriefed the relevant discussions at LPC to bring everybody up to 
speed.  Bharata said the clearest outcome for him was to separate out the 
asynchronous promotion mechanism and establish its independent value.  
After that, sources of page hotness and hardware specific mechanisms could 
be integrated into the kernel.  Wei suggested getting the asynchronous 
promotion mechanism merged soon so that other things can build off of it.  
Bharata said he will try to post the next revision before the next 
meeting.  Gregory noted that there may be some pushback on asynchronous 
promotion with regards to cgroup charging of the cycles.

Davidlohr noted that this matched his expectation and that he is working 
on upstreaming the CHMU support as a CXL driver, currently using DAMON as 
its backend.  He'll then wait to see the outcome of the other patch series 
in parallel.  Jonathan questioned how integration with DAMON would be done 
to eventually support the asynchronous promotion stuff.  Davidlohr said 
this would be done in the most generic way as possible.  CHMU has a 
natural interface with ranges and large continguous chunks of memory that 
can be passed to any interface.  If pghot is confined to the NUMA 
Balancing aspects, then CHMU wouldn't be integrated with it.  Jonathan 
asked if we're really getting large contiguous chunks of memory from the 
CHMU and Davidlohr said this is what he is seeing in his evaluation.

Bharata suggested it may be difficult to get the configuration of DAMON 
correct, this is a complex undertaking.  If the only way to use CHMU is 
through DAMON, this may be challenging.  Davidlohr agreed the 
configuration was more complex but if you're going to go out and buy a 
CHMU supported device then you can likely configure this correctly.  Wei 
asked if the CHMU is implemented as a driver and Davidlohr confirmed that 
it was; in this case, any backend could likely use it.  Davidlohr said he 
would send out the CHMU series and then extend it through sysfs before 
adding the missing pieces in DAMON.

Gregory made an interface that took a list of physical addresses or 
folios, the idea being that a driver or external service would send CRAM 
information on the memory to promote.  There didn't end up being an 
important reason to only be in CRAM, we could implement support for 
private nodes.  For normal memory tiers that happen to be on private 
nodes, the hotness becomes important and can require additional plumbing 
that will evolve over time.  This will become more obvious as private 
nodes moves out of RFC and into a formal proposal.

Gregory updated on the CRAM discussions as LPC.  He had a lot of 
discussions and there was general openness to the first user being HBM as 
the minimal upstreamable place.  There isn't a big leap between this and 
CRAM.  Jonathan asked how hotness tracking is being done for CRAM today, 
and Gregory said it can technically support NUMA Balancing and another 
approach is pushed off to the driver by telling it what to promote.  He'll 
post the next revision of private nodes after the merge window followed by 
CRAM.

----->o-----
Huan updated that he had sent patches to Yiannis for supporting 
non-temporal stores and memory error handling.  He will post these to the 
upstream mailing list soon.  It supports folios, not just 4KB sizes.  
There are some changes to the interfaces that are in flight that may 
result in some changes.

----->o-----
Shivank discussed two concerns for page migration hardware offload: (1) 
there's an assumption that SDXI would be idle so there's a concern about 
other processes using it -- there may be competition between kernel and 
user, (2) if batching folios for migration, there may be latency before 
the first folio is available.  For this, we could use a migration mode 
that is not sensitive to latency.  He is planning on getting the easier 
bits merged first.

----->o-----
Joshua noted that he was going to follow up on his tier aware patches 
after the merge window and requested that anybody with use cases for it 
would express that on the upstream mailing list.

----->o-----
NOTE!!!  The next meeting will be after Daylight Savings Time ends, so
the meeting will be one hour later if your time zone does not make the
switch.

Next meeting will be on Thursday, November 5 at 8:30am PST (UTC-8),
everybody is welcome: https://meet.google.com/jak-ytdx-hnm

Topics for the next meeting:

 - update on Bharata's asynchronous promotion mechanism separated out
   from the rest of pghot, targeting posting the revision before the next
   meeting
 - discuss current status of CHMU driver and its integration with DAMON
   or any other generic backend as needed
 - update on private node support as well as CRAM to be posted after the
   merge window closes
 - update on combined patch series for supporting non-temporal stores in
   migrate_pages() with memory error handling (series from Yiannis +
   Huan)
 - discuss Shivank's series for enlightening migrate_pages() for hardware
   assists and competition between the kernel and user use cases
 - update on Bharata's support for sources of page hotness and hardware
   support
 - Teja's update on SDXI page migration based on AMD patches and hardware
   issues being encountered that do not result in page migration
 - update on tier-aware memcg limits status and production testing based
   on the latest major overhaul
 - first class support for virtualization based memory tier support, how
   to leverge memory tiers in the guest

Please let me know if you'd like to propose additional topics for
discussion, thank you!

Re: [Linux Memory Hotness and Promotion] Notes from October 22, 2026

[SJ Park]

Hi David,


Thank you for sharing these nice notes!  I unforunately keep being unable to
join the meeting, so these notes are very helpful.

On Fri, 23 Oct 2026 20:14:53 -0700 (PDT) David Rientjes <rientjes@google.com> wrote:

> Hi everybody,
> 
> Here are the notes from the last Linux Memory Hotness and Promotion call
> that happened on Thursday, October 22.  Thanks to everybody who was 
> involved!
> 
> These notes are intended to bring people up to speed who could not attend 
> the call as well as keep the conversation going in between meetings.
[...]
> Davidlohr noted that this matched his expectation and that he is working 
> on upstreaming the CHMU support as a CXL driver, currently using DAMON as 
> its backend.  He'll then wait to see the outcome of the other patch series 
> in parallel.  Jonathan questioned how integration with DAMON would be done 
> to eventually support the asynchronous promotion stuff.  Davidlohr said 
> this would be done in the most generic way as possible.  CHMU has a 
> natural interface with ranges and large continguous chunks of memory that 
> can be passed to any interface.  If pghot is confined to the NUMA 
> Balancing aspects, then CHMU wouldn't be integrated with it.  Jonathan 
> asked if we're really getting large contiguous chunks of memory from the 
> CHMU and Davidlohr said this is what he is seeing in his evaluation.

Makes sense to me.  If CHMU can find address ranges of hot memory, it could
directly ask DAMOS to migrate those to a specific node.  DAMOS does not provide
such interface for now.  We could implement new API like damos_add_folios(),
which I proposed at LSFMMBPF'25.  For CHMU, I think it could be easier if the
API receives address ranges instead of folios.

Yet another idea is let users ask DAMON to do everything.  That is, DAMON will
turn on CHMU, get the hot memory information from it, and do the migration all
by itself.  DAMON sysfs ABI would need to be extended for CHMU configuration.

Both look doable to me.  Davidlohr may have a much better idea.  In any case, I
will be happy to help as always.  Please don't hesitate to reach out even if it
is only for internal prototyping that doesn't pursue upstreaming.

> 
> Bharata suggested it may be difficult to get the configuration of DAMON 
> correct, this is a complex undertaking.  If the only way to use CHMU is 
> through DAMON, this may be challenging.  Davidlohr agreed the 
> configuration was more complex but if you're going to go out and buy a 
> CHMU supported device then you can likely configure this correctly.

We (DAMON community) provide and support a auto-tuning based DAMON
configuration for general memory tiering on 2 nodes setup [1].  The support
means that if a user tried it and find it doesn't work as expected, they can
reach out to DAMON community for help.  In the case, we will help investigating
the issue and provide solutions.  We also merge back the solution to the
configuration and DAMON itself.

Of course our support is not constrained to only the auto-tuning based
configuration.  We samely help for any custom configuration usages.

Some people indeed trying the auto-tuning based config and reaching out to us
in the way.  We are helping them and improve DAMON.  I believe it will be
similar for CHMU case.

DAMON is an open source project that is actively maintained, supported and open
to changes.  Please feel free to ask questions and changes.

[1] https://github.com/damonitor/damo/blob/next/scripts/mem_tier.sh


Thanks,
SJ

[...]