[Linux Memory Hotness and Promotion] Notes from August 27, 2026

[David Rientjes <rientjes@google.com>]

Hi everybody,

Here are the notes from the last Linux Memory Hotness and Promotion call
that happened on Thursday, August 27.  Thanks to everybody who was 
involved!

These notes are intended to bring people up to speed who could not attend 
the call as well as keep the conversation going in between meetings.

----->o-----
While Yiannis is on leave, I noted that Huan from Google would pursue 
sending the non-temporal stores patches upstream that include his 
extensions for handling of memory errors.  They will sync together before 
the upstream posting.

----->o-----
Bharata updated that he is getting ready for pghot v9.  There were a lot 
of corner cases addressed in preparation for this.  He also tried to 
compare DAMON with pghot and was able to get some realistic DAMON 
configuration working after some iterations.  He posted some initial 
comparison numbers and more will follow in the pghot v9 posting.

He's also planning on sepraating out the pghot-hwhints (IBS Memory 
Profiler source) into a separate patch series.

Hrushikesh is working on reviving the PTE A bit based source (kscand) that 
Raghu worked on earlier; the plan is to integrate it into pghot, get 
benchmark results, and then post it.

Bharata continued to request additional patch review from the group and 
any insight into relevant benchmarks would be very useful.

----->o-----
Shivank noted that there were no new updates at this point for his 
migrate_pages() series because work was being done on the rmap series.  He 
is waiting for additional feedback upstream and preparing for a discussion 
at LPC.  He is also focused on migrate_pages() refactoring to make this 
cleaner, including for pending batching optimizations.

Teja discussed hardware errors that were being encountered in his testing 
of SDXI, he was not yet able to get pages to be migrated.  He is iterating 
on this with AMD and will continue conversations next week to try to make 
forward progress.

----->o-----
Wei Xu suggested we add an additional topic for this meeting series: 
compression enlightenment for memory tiered architectures.  He was 
advocating for a unified approach upstream for handling of second tier 
compression.  Specifically, he was looking at software support for 
efficiency such as handling of free memory -- the goal is to represent 
this with no memory footprint in the second compressed tier.  This would 
likely require hardware signals to indicate.  Secondly, he noted that the 
physical capacity of memory on the second tier may be insufficient if the 
compression ratio is not good; we want to ensure that we do not run out of 
memory in these cases perhaps with watermark handling.

He also noted that handling MCEs for compression support does not have 
first-class support in the upstream kernel that we may want to improve.  
Gregory noted that we may end up killing critical processes and that it 
may be unrecoverable.  Gregory suggested that we cannot allow writes to go 
to the device uncontended although there would be performance implications 
of this.  Gregory has an end-to-end configuration for this already in 
progress including separating the second tier into compressed and 
uncompressed memory to provide the necessary reliability while still 
having memory efficiency.  The most obvious example was the lack of 
control over file writes.  We will invite others from Google to attend the 
next instance of this meeting for more discussion.

Gregory noted that he has a talk for LPC scheduled for compressed RAM that 
will cover his end-to-end design with performance numbers using qemu.  We 
decided to dedicate the instance of this meeting in two weeks to this 
topic.

----->o-----
Gregory updated on behalf of Joshua for the tier-aware memcg limits work.  
He noted that the LRU inversion issue was due to the system as a whole 
being pressured.  They've concluded that there is no good solution -- when 
we are going to swap, wherever you can get the memory is probably the best 
we can do.  Aggressive demotion typically just ends up resulting in much 
worse performance.  For tier-aware memcg limits itself, there is a major 
versioning change happening for this.  When testing previous iterations, 
there were issues when the amount of hot memory was larger than the top 
tier.  They have a solid solution that provides fairness using memory.max, 
high, low, and min settings.  The proportions that you get are based on 
the physical topology of memory tiers on the system itself (3:1 top tier 
to CXL provides 3:1 ratios for each memcg).

The latest revision is going through production testing.  Notably, there 
are no new tunables to configure.  This is enabled through a kernel boot 
parameter, mostly for production testing.  It also helps to avoid adding 
new userspace APIs while still iterating through solutions.  Later, this 
could be enabled at runtime.  Since we're in production testing right now, 
it's possible that this will move beyond RFC in the near term.

----->o-----
Next meeting will be on Thursday, September 10 at 8:30am PDT (UTC-7),
everybody is welcome: https://meet.google.com/jak-ytdx-hnm

Topics for the next meeting:

 - Gregory's end-to-end compressed RAM design and performance numbers
   obtained with qemu experimentation
 - update on combined patch series for supporting non-temporal stores in
   migrate_pages() with memory error handling (series from Yiannis +
   Huan)
 - v9 of pghot and the PTE A bit based source (kscand) for inclusion in
   the upstream kernel
 - v6 of Shivank's series for enlightening migrate_pages() for hardware
   assists and his rmap batch series and LPC discussion
 - Teja's update on SDXI page migration based on AMD patches and hardware
   issues being encountered that do not result in page migration
 - update on tier-aware memcg limits status and production testing based
   on the latest major overhaul
 - first class support for virtualization based memory tier support, how
   to leverge memory tiers in the guest
 - discuss generalized subsystem for providing bandwidth information
   independent of the underlying platform, ideally through resctrl,
   otherwise utilizing bandwidth information will be challenging
   + preferably this bandwidth monitoring is not per NUMA node but rather
     slow and fast

Please let me know if you'd like to propose additional topics for
discussion, thank you!