[PATCH] mm/memory_hotplug: cache zone stats for "auto-movable" online policy

[PATCH] mm/memory_hotplug: cache zone stats for "auto-movable" online policy

[Zhijian Han]

When onlining memory using the "auto-movable" online policy with
nid == NUMA_NO_NODE, auto_movable_can_online_movable() walks all
populated zones across all nodes on each invocation to collect the
MOVABLE vs. KERNEL_EARLY stats.  This walk happens twice for every
memory block onlining decision (global check and, with NUMA awareness
enabled, per-node check) and is repeated for every single memory block
that gets onlined, although the underlying zone counters rarely change.

Cache the stats for nid == NUMA_NO_NODE and only recalculate them when
a zone counter actually changes.  All modifications of the relevant
zone counters (present_pages, present_early_pages) funnel through
adjust_present_page_count(), which is only called while holding the
mem_hotplug_lock in write mode and the device_lock() of the memory
block device.  Invalidate the cache there, such that the next
auto-movable onlining decision recalculates the stats from scratch.

CMA adjustments (zone->cma_pages) only happen during boot (via
init_cma_reserved_pageblock()/init_cma_pageblock(), both __init) and
cannot race with memory onlining.  Boot-time zone initialization
happens before any memory block can be onlined.

The per-node path (nid != NUMA_NO_NODE) is left uncached; it only
walks a single node's zones.

Resolve a TODO that was left when the "auto-movable" online policy was
introduced.

Tested on QEMU (x86_64, 512M boot + 256M hotplugged pc-dimm,
online_policy=auto-movable): hotplugged memory blocks get onlined to
ZONE_MOVABLE as expected, and an offline/online cycle of a hotplugged
block shows the correct zone counters after cache invalidation, with
no kernel warnings.

Link: https://lore.kernel.org/r/20210806124715.17090-3-david@redhat.com
Signed-off-by: Zhijian Han <hanzhijian1991@gmail.com>
---
 mm/memory_hotplug.c | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index d7a59167bec4..dfac59739a6d 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -789,6 +789,19 @@ struct auto_movable_stats {
 	unsigned long movable_pages;
 };
 
+/*
+ * Cached stats for all populated zones across all nodes. Modified (incl.
+ * invalidation) only while holding the mem_hotplug_lock in write mode and
+ * the device_lock() of the memory block device.
+ */
+static struct auto_movable_stats auto_movable_zone_stats;
+static bool auto_movable_zone_stats_valid;
+
+static void auto_movable_zone_stats_invalidate(void)
+{
+	auto_movable_zone_stats_valid = false;
+}
+
 static void auto_movable_stats_account_zone(struct auto_movable_stats *stats,
 					    struct zone *zone)
 {
@@ -849,9 +862,14 @@ static bool auto_movable_can_online_movable(int nid, struct memory_group *group,
 
 	/* Walk all relevant zones and collect MOVABLE vs. KERNEL stats. */
 	if (nid == NUMA_NO_NODE) {
-		/* TODO: cache values */
-		for_each_populated_zone(zone)
-			auto_movable_stats_account_zone(&stats, zone);
+		if (auto_movable_zone_stats_valid) {
+			stats = auto_movable_zone_stats;
+		} else {
+			for_each_populated_zone(zone)
+				auto_movable_stats_account_zone(&stats, zone);
+			auto_movable_zone_stats = stats;
+			auto_movable_zone_stats_valid = true;
+		}
 	} else {
 		for (i = 0; i < MAX_NR_ZONES; i++) {
 			pg_data_t *pgdat = NODE_DATA(nid);
@@ -1079,6 +1097,8 @@ void adjust_present_page_count(struct page *page, struct memory_group *group,
 	zone->present_pages += nr_pages;
 	zone->zone_pgdat->node_present_pages += nr_pages;
 
+	auto_movable_zone_stats_invalidate();
+
 	if (group && movable)
 		group->present_movable_pages += nr_pages;
 	else if (group && !movable)
-- 
2.43.0

Re: [PATCH] mm/memory_hotplug: cache zone stats for "auto-movable" online policy

[David Hildenbrand (Arm)]

On 9/22/26 10:22, Zhijian Han wrote:
> When onlining memory using the "auto-movable" online policy with
> nid == NUMA_NO_NODE, auto_movable_can_online_movable() walks all
> populated zones across all nodes on each invocation to collect the
> MOVABLE vs. KERNEL_EARLY stats.  This walk happens twice for every
> memory block onlining decision (global check and, with NUMA awareness
> enabled, per-node check) and is repeated for every single memory block
> that gets onlined, although the underlying zone counters rarely change.
> 
> Cache the stats for nid == NUMA_NO_NODE and only recalculate them when
> a zone counter actually changes.  All modifications of the relevant
> zone counters (present_pages, present_early_pages) funnel through
> adjust_present_page_count(), which is only called while holding the
> mem_hotplug_lock in write mode and the device_lock() of the memory
> block device.  Invalidate the cache there, such that the next
> auto-movable onlining decision recalculates the stats from scratch.
> 
> CMA adjustments (zone->cma_pages) only happen during boot (via
> init_cma_reserved_pageblock()/init_cma_pageblock(), both __init) and
> cannot race with memory onlining.  Boot-time zone initialization
> happens before any memory block can be onlined.
> 
> The per-node path (nid != NUMA_NO_NODE) is left uncached; it only
> walks a single node's zones.
> 
> Resolve a TODO that was left when the "auto-movable" online policy was
> introduced.

I have to ask

1) Why is this TODO a concern for you. Are you a frequent memory hotplug user or
   somehow working on memory hotplug.

2) Did you use an LLM to come up with this idea or this patch.

3) Why is there no discussion / performance measurements whether the TODO should
be removed instead.

I'll point you at:
https://lore.kernel.org/linux-mm/20251206212507.135503-1-swarajgaikwad1925@gmail.com/


-- 
Cheers,

David