Re: [PATCH RFC v2 0/3] Demote to lower tier using non-temporal stores

[Gregory Price <gourry@gourry.net>]

On Thu, Jul 30, 2026 at 05:02:56PM +0200, Yiannis Nikolakopoulos wrote:
> In most memory tiering scenarios, the memory to be demoted is expected
> to be cold and most likely out of the node's last-level cache (as well
> as target pages in the target node). Using non-temporal stores instead
> of a standard memcpy path can reduce the cache pollution in the local
> node and the bandwidth overhead to the target node. Furthermore, for
> certain types of CXL devices that support in-line memory compression,
> the last-level cache eviction patterns can negatively affect the
> bandwidth of the device. Non-temporal stores can mitigate this.
> 
> This patch-set introduces a new migrate_mode flag for using non-temporal
> stores that is used only in the demotion path. Patch 1 adds some helpers in
> x86 and mm to bring non-temporal stores support to a respective folio_copy
> function. Patch 2 adds the new flag and necessary changes for compatibility
> with the existing behavior. Patch 3 uses the new flag for demotions.
> 
> Experimental data: in a CXL system with 1 memory expander, a microbenchmark
> that allocates N=64 GB memory in the local node and then triggers demotion
> using memory.reclaim, shows a practically complete elimination of read
> traffic on the device, i.e. write traffic is N GB with and without the
> patch, while read traffic drops from N to almost 0 with the patch.
> 
> Opens:
> 1. There is some "duplication" in the x86 tree and a bit in mm. Can we do
>    something better there? As it is now in copy_mc_to_kernel_nt we
> duplicate the machine check functionality, which if available will override
> the non-temporal. We were not sure how to prioritize these two and what's
> the best approach here. Can we completely skip the machine checked for this
> path? Huan Nguyen has some ideas here that we will align for the next
> version.
> 2. I am not sure how this should be structured so that it is easily
> adopted in other architecture trees (e.g. aarch64). We rely on
> memcpy_flushcache for x86_64 but this does not use non temporal stores
> in ARM. ARM support is currently out of our scope but any input is
> appreciated.
> 

I'm still a bit confused why using NT Stores needs to be an explicit
option - rather than the default behavior if NT Store is available.

Lets assume we did this for all ASYNC requests, is there a negative
effect? A positive effect? Why the new ASYNC type? Is there a
correctness issue?

~Gregory