[Linux Memory Hotness and Promotion] Notes from September 24, 2026

[Linux Memory Hotness and Promotion] Notes from September 24, 2026

[David Rientjes]

Hi everybody,

Here are the notes from the last Linux Memory Hotness and Promotion call
that happened on Thursday, September 24.  Thanks to everybody who was 
involved!

These notes are intended to bring people up to speed who could not attend 
the call as well as keep the conversation going in between meetings.

----->o-----
Bharata updated on the status of his work.  He posted pghot-hwhints for 
the IBS Memory Profiler as a separate but dependent patch series as it is 
x86 specific and the pghot patch series is becoming too big with multiple 
sources of hotness information posted together.

Bharata noted that the benchmark numbers for pghot-hwhints so far looks 
promising and that he is looking forward to discussing this topic at LPC 
coming up.

----->o-----
Huan updated on the status of his work with Yiannis for non-temporal 
stores in migrate_pages(), he prepared a patch series to send for feedback 
to the group.  He looked into Shivank's feedback and work being done for 
batching with 2MB pages.  Today, we need to do a loop for each 4KB page, 
Shivank's work speeds this up but Huan has not yet integrated this since 
Shivank's work is not yet upstream.  There are also mixed results posted 
upstream.  Thus, for now this is being considered orthogonally until 
Shivank's changes land upstream.

I asked if Meta was also going to be looking into these optimizations; 
Gregory noted that they'll be waiting for it to land upstream.  They've 
been looking at basic correctness of NUMA Balancing rather than 
optimizations for now.  Per his discussion upstream, it turns out that 
NUMA Balancing, at least for tiering, has been broken for the last four 
years.

Shivank updated on his patches[1] for non-temporal stores without 
overloading migrate_mode.  Feedback on this would be useful.  He will also 
be presenting at LPC including additional experiments that have been run.

----->o-----
Davidlohr expressed concern about the direction of pghot overall.  He 
noted the upstream discussion has focused on not having any regressions, 
but there isn't an obvious gain either.  It looks like Joshua had the same 
conclusion although PSI may have improved.  Davidlohr has been working on 
integrating CHMU into pghot and found the experience to be complicated so 
he ended up bypassing most of it.  Whenever he got an interrupt from the 
device, he would try to create as many contiguous HPA ranges as he could 
to feed to pghot; once he got that, he decided to promote instead of doing 
decay and got a 3x improvement over NUMA Balancing that he was expecting.  
He suggested we may want to go back to the drawing board with regard to 
the complexity and interfaces of pghot.

Davidlohr suggested we may want to rely more on user-driven proactive 
reclaim and can then tune CHMU as desired so there is some control over 
freeing and promotion.  Another option would be to integrate this into 
DAMON as well.  One concern is that if this is integrated into pghot then 
we lose locality information.

Gregory said that pghot provides value in doing this asynchronously, which 
NUMA Balancing does not do, and that avoids lengthy stalled faults.  In 
this case, Davidlohr suggested we may just punt it to a kthread that would 
recheck if it's hot and promote.  Gregory thinks we're proposing 
significant complexity in pghot that doesn't yet carry all its weight but 
if we were to focus on the async part then this could be a start and we 
could build incrementally on top.  Davidlohr said we're still losing 
locality information; Gregory suggested this is a general limitation of 
tiering in general.  Davidlohr said we could feed the locality information 
to the kthread although this would include overhead for tracking.

Davidlohr agreed with the value of the asynchronous promotion handling, 
including for NUMA Balancing, but not with catering to IBS necessarily.  
Gregory is a proponent of splitting out kpromoted from the rest of the 
system.

Wei agreed with Gregory's assessment although the data structures are up 
for debate.  We could iterate on the data structures for this.  DAMON has 
a ton of additional support and we need to discuss what all comes along 
with it if this is a general tiering solution.  Gregory suggested the 
asynchronous promotion work could also be used by DAMON.

Joshua broke down the components of pghot: device drivers, kscand, 
kmigrated, and pghot interface.  If they get separated, it would also be 
useful for code reviews.  He was concerned that pghot may be heading in 
the same direction as DAMON.  Davidlohr agreed with this.

Bharata, have you thought about the possibility of only introducing the 
asynchronous promotion support first, getting alignment from stakeholders 
on that, and then extending pghot with additional enhancements later?

----->o-----
NOTE!!!  The next meeting will be canceled due to LPC 2026.

Next meeting will be on Thursday, October 22 at 8:30am PDT (UTC-7),
everybody is welcome: https://meet.google.com/jak-ytdx-hnm

Topics for the next meeting:

 - debrief discussions at LPC
 - directional discussion for pghot and how to build it incrementally:
   focus on async promotion mechanisms first and then add enhancements on
   top of this separately
 - update on combined patch series for supporting non-temporal stores in
   migrate_pages() with memory error handling (series from Yiannis +
   Huan) for 4KB, extensions for 2MB later from Shivank
 - v6 of Shivank's series for enlightening migrate_pages() for hardware
   assists and his rmap batch series as well as non-temporal stores 
   without overloading migrate_mode
 - Teja's update on SDXI page migration based on AMD patches and hardware
   issues being encountered that do not result in page migration
 - update on tier-aware memcg limits status and production testing based
   on the latest major overhaul
 - first class support for virtualization based memory tier support, how
   to leverge memory tiers in the guest
 - discuss generalized subsystem for providing bandwidth information
   independent of the underlying platform, ideally through resctrl
   + preferably this bandwidth monitoring is not per NUMA node but rather
     slow and fast

Please let me know if you'd like to propose additional topics for
discussion, thank you!

[1]
https://lore.kernel.org/linux-mm/20260902-migrate-refactor-shivank-v1-9-9dcca87669c4@amd.com/ 

Re: [Linux Memory Hotness and Promotion] Notes from September 24, 2026

[Bharata B Rao]

On 27-Sep-26 3:13 AM, David Rientjes wrote:
> 
> Bharata, have you thought about the possibility of only introducing the 
> asynchronous promotion support first, getting alignment from stakeholders 
> on that, and then extending pghot with additional enhancements later?

I have gone down that route earlier [1] but we can always revisit and resurrect
the effort if required.

As discussed during the call, I think it would be good to have a dedicated F2F
meeting or BoF during LPC to reach to some conclusion/consensus.

[1] https://lore.kernel.org/linux-mm/20250616133931.206626-1-bharata@amd.com/#t

Regards,
Bharata.