[RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Sourav Panda]

Overview
This patch series introduces a dynamic, NUMA-aware HugePage Cache,
backed by a kernel shrinker to safely return memory under pressure, and
integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
specifically targeting gigantic (1GB) hugepages. The goal is to solve
the tradeoff between allocation latency and memory
fungibility in virtualized and heterogeneous cloud environments.

---

The Core Problem: Allocation Latency vs. Memory Fungibility

With highly heterogeneous workloads, latency-critical applications demand
gigantic hugepages. However, dynamic runtime allocation of 1GB pages
from the buddy allocator (via CMA) is slow.

To bypass this latency, operators often pre-allocate hugepages
statically. However, this locks up the memory: when the HugeTLB
workloads are idle, that memory is completely unavailable for other
buddy-allocator workloads (e.g., page cache, anonymous memory). If buddy
memory is exhausted, the system will OOM even if gigabytes of HugeTLB
pages are sitting idle.

This series resolves this challenge by delivering Dynamic Fungibility:

1.  Dynamic Caching: Intercepts freed surplus hugepages and recycles
    them into a NUMA-aware cache instead of dissolving them immediately.
2.  Fast Allocations: Satisfies subsequent dynamic allocations
    instantly from this warm, local hugepage cache.
3.  Kernel Shrinker Integration: Registers a NUMA-aware kernel shrinker
    to dynamically dissolve cached pages back to the buddy allocator
    under memory pressure, restoring host/guest memory fungibility.
4.  Free Page Reporting Integration: For virtualized environments (Guest
    VMs), cached pages trigger background Free Page Reporting via
    virtio-balloon. This allows the host to reclaim the physical memory
    while the guest retains its Vmemmap Optimization (HVO) metadata
    savings (~14GB saved per 1TB VM)!

  +---------------+   Slow Allocate  +---------------------+
  | Buddy         | ---------------> | Active HugeTLB Page |
  | Allocator     |                  +---------------------+
  +---------------+                    |                ^
    ^                                  | (1) Free to    | (2) Fast
    | (3) Under                        v     Cache      |     Allocate
    | Pressure                       +---------------------+
    | Shrink()  <------------------- | HugeTLB Cache       |
                                     +---------------------+
                                       |
                                       v (4) Free Page Reporting
                                         (Host Reclaim in the case
                                          of virtualization)

---

Patch Series Structure

Patch 1/6: mm/hugetlb: add Kconfig and basic cache infrastructure
  - Introduces the CONFIG_HUGETLB_CACHE option, hstate tracking fields,
    and the HPG_cached page flag.
  - Establishes the clean helper API (hugetlb_folio_is_cached(),
    hugetlb_cache_remove(), hugetlb_cache_add()) and updates
    remove_hugetlb_folio() to be cache-aware, eliminating inline #ifdef
    blocks.
Patch 2/6: mm/hugetlb: implement cache recycling and allocation
  - Hooks up recycling in free_huge_folio() (up to the cache limit) and
    allocation in alloc_surplus_hugetlb_folio().
  - Implements MRU allocation policy for maximum warmth, poison safety
    checks, and MTE/dcache cleaning.
Patch 3/6: mm/hugetlb: add sysfs interfaces for cache
  - Exposes global and per-node sysfs attributes (max_cached_huge_pages,
    nr_cached_hugepages) for dynamic userspace control, including NUMA
    memory policy scaling.
  - Supports dynamic delta adjustments (+1/-1) to safely scale cache
    sizes alongside concurrent background reclaim operations.
Patch 4/6: mm/hugetlb: add memory shrinker for cache
  - Registers a NUMA-aware kernel shrinker to evict and dissolve cached
    gigantic pages back to buddy under memory pressure.
Patch 5/6: Documentation/admin-guide/mm/hugetlbpage.rst: document cache
    interfaces
  - Documents the Kconfig option, sysfs attributes, and shrinker
    behavior in the admin guide.
Patch 6/6: mm/hugetlb: support free page reporting for cached hugepages
  - Integrates the cache with the Free Page Reporting framework (virtio-
    balloon), introducing the HPG_reported flag and the
    reporting/isolation/draining lifecycle.
Sourav Panda (6):
  mm/hugetlb: add Kconfig and basic cache infrastructure
  mm/hugetlb: implement cache recycling and allocation
  mm/hugetlb: add sysfs interfaces for cache
  mm/hugetlb: add memory shrinker for cache
  Documentation/admin-guide/mm/hugetlbpage.rst: document cache
    interfaces
  mm/hugetlb: support free page reporting for cached hugepages

 Documentation/admin-guide/mm/hugetlbpage.rst |  34 +-
 fs/Kconfig                                   |   9 +
 include/linux/hugetlb.h                      |  45 ++
 include/linux/page_reporting.h               |   1 +
 mm/hugetlb.c                                 | 590 ++++++++++++++++++-
 mm/hugetlb_internal.h                        |   9 +
 mm/hugetlb_sysfs.c                           | 158 +++++
 mm/page_reporting.c                          |  10 +-
 mm/page_reporting.h                          |   6 +
 9 files changed, 844 insertions(+), 18 deletions(-)

-- 
2.55.0.rc0.799.gd6f94ed593-goog

[PATCH 1/6] mm/hugetlb: add Kconfig and basic cache infrastructure

[Sourav Panda]

Introduce CONFIG_HUGETLB_CACHE Kconfig option and add the basic
infrastructure for the HugeTLB dynamic cache.

This includes:
- Adding new fields to struct hstate to track cached pages
  (nr_cached_hugepages, max_cached_huge_pages, and node-specific versions)
  and initializing them.
- Introducing a new hugetlb page flag HPG_cached to track cached folios.
- Updating remove_hugetlb_folio() to be cache-aware: if the folio has
  HPG_cached set, decrement cache counters instead of the free page
  counters. This prevents leaks during memory hotplug
  (dissolve_free_hugetlb_folios).
- Preventing inflation of free_huge_pages by cached pages to protect
  memory reservation guarantees.

Signed-off-by: Sourav Panda <souravpanda@google.com>
---
 fs/Kconfig              |  9 +++++
 include/linux/hugetlb.h | 14 +++++++
 mm/hugetlb.c            | 82 +++++++++++++++++++++++++++++++++++++----
 3 files changed, 98 insertions(+), 7 deletions(-)

diff --git a/fs/Kconfig b/fs/Kconfig
index 43cb06de297f..d33d1973ce8f 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -268,6 +268,15 @@ config HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON
 	  The HugeTLB Vmemmap Optimization (HVO) defaults to off. Say Y here to
 	  enable HVO by default. It can be disabled via hugetlb_free_vmemmap=off
 	  (boot command line) or hugetlb_optimize_vmemmap (sysctl).
+
+config HUGETLB_CACHE
+	bool "HugeTLB dynamic cache"
+	help
+	  Enables a dynamic, NUMA-aware hugepage cache for
+	  gigantic hugepages to allow faster allocation of surplus pages.
+	  Surplus pages are recycled into this cache upon release
+	  instead of being freed back to the buddy allocator immediately.
+	  Subsequent hugetlb allocations will prefer this cache.
 endif # HUGETLBFS
 
 config HUGETLB_PAGE
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 2abaf99321e9..4768f52ddd35 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -585,6 +585,9 @@ enum hugetlb_page_flags {
 	HPG_vmemmap_optimized,
 	HPG_raw_hwp_unreliable,
 	HPG_cma,
+#ifdef CONFIG_HUGETLB_CACHE
+	HPG_cached,
+#endif
 	__NR_HPAGEFLAGS,
 };
 
@@ -645,6 +648,9 @@ HPAGEFLAG(Freed, freed)
 HPAGEFLAG(VmemmapOptimized, vmemmap_optimized)
 HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable)
 HPAGEFLAG(Cma, cma)
+#ifdef CONFIG_HUGETLB_CACHE
+HPAGEFLAG(Cached, cached)
+#endif
 
 #ifdef CONFIG_HUGETLB_PAGE
 
@@ -670,6 +676,14 @@ struct hstate {
 	unsigned int nr_huge_pages_node[MAX_NUMNODES];
 	unsigned int free_huge_pages_node[MAX_NUMNODES];
 	unsigned int surplus_huge_pages_node[MAX_NUMNODES];
+
+#ifdef CONFIG_HUGETLB_CACHE
+	unsigned long nr_cached_hugepages;
+	unsigned int nr_cached_hugepages_node[MAX_NUMNODES];
+	struct list_head hugepage_cache_lists[MAX_NUMNODES];
+	unsigned long max_cached_huge_pages;
+	unsigned int max_cached_huge_pages_node[MAX_NUMNODES];
+#endif
 	char name[HSTATE_NAME_LEN];
 };
 
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index c921287489de..15be3cf54606 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1381,6 +1381,60 @@ static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask, int
 }
 #endif
 
+#ifdef CONFIG_HUGETLB_CACHE
+static inline bool hugetlb_folio_is_cached(struct folio *folio)
+{
+	return folio_test_hugetlb_cached(folio);
+}
+
+static void hugetlb_cache_add(struct hstate *h, struct folio *folio, bool from_surplus)
+{
+	int nid = folio_nid(folio);
+
+	list_move_tail(&folio->lru, &h->hugepage_cache_lists[nid]);
+	folio_set_hugetlb_freed(folio);
+	folio_set_hugetlb_cached(folio);
+	h->nr_cached_hugepages++;
+	h->nr_cached_hugepages_node[nid]++;
+	if (from_surplus) {
+		h->surplus_huge_pages--;
+		h->surplus_huge_pages_node[nid]--;
+	}
+}
+
+static void hugetlb_cache_remove(struct hstate *h, struct folio *folio, bool to_surplus)
+{
+	int nid = folio_nid(folio);
+
+	list_del_init(&folio->lru);
+	folio_clear_hugetlb_freed(folio);
+	folio_clear_hugetlb_cached(folio);
+	h->nr_cached_hugepages--;
+	h->nr_cached_hugepages_node[nid]--;
+	if (to_surplus) {
+		h->surplus_huge_pages++;
+		h->surplus_huge_pages_node[nid]++;
+	}
+}
+#else
+static inline bool hugetlb_folio_is_cached(struct folio *folio)
+{
+	return false;
+}
+
+static inline void hugetlb_cache_add(struct hstate *h,
+				     struct folio *folio,
+				     bool from_surplus)
+{
+}
+
+static inline void hugetlb_cache_remove(struct hstate *h,
+					struct folio *folio,
+					bool to_surplus)
+{
+}
+#endif
+
 /*
  * Remove hugetlb folio from lists.
  * If vmemmap exists for the folio, clear the hugetlb flag so that the
@@ -1401,12 +1455,16 @@ void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
 	if (hstate_is_gigantic_no_runtime(h))
 		return;
 
-	list_del(&folio->lru);
-
-	if (folio_test_hugetlb_freed(folio)) {
-		folio_clear_hugetlb_freed(folio);
-		h->free_huge_pages--;
-		h->free_huge_pages_node[nid]--;
+	if (hugetlb_folio_is_cached(folio)) {
+		if (folio_test_hugetlb_freed(folio))
+			hugetlb_cache_remove(h, folio, false);
+	} else {
+		list_del(&folio->lru);
+		if (folio_test_hugetlb_freed(folio)) {
+			folio_clear_hugetlb_freed(folio);
+			h->free_huge_pages--;
+			h->free_huge_pages_node[nid]--;
+		}
 	}
 	if (adjust_surplus) {
 		h->surplus_huge_pages--;
@@ -4169,8 +4227,18 @@ void __init hugetlb_add_hstate(unsigned int order)
 	__mutex_init(&h->resize_lock, "resize mutex", &h->resize_key);
 	h->order = order;
 	h->mask = ~(huge_page_size(h) - 1);
-	for (i = 0; i < MAX_NUMNODES; ++i)
+	for (i = 0; i < MAX_NUMNODES; ++i) {
 		INIT_LIST_HEAD(&h->hugepage_freelists[i]);
+#ifdef CONFIG_HUGETLB_CACHE
+		INIT_LIST_HEAD(&h->hugepage_cache_lists[i]);
+		h->nr_cached_hugepages_node[i] = 0;
+		h->max_cached_huge_pages_node[i] = 0;
+#endif
+	}
+#ifdef CONFIG_HUGETLB_CACHE
+	h->nr_cached_hugepages = 0;
+	h->max_cached_huge_pages = 0;
+#endif
 	INIT_LIST_HEAD(&h->hugepage_activelist);
 	snprintf(h->name, HSTATE_NAME_LEN, "hugepages-%lukB",
 					huge_page_size(h)/SZ_1K);
-- 
2.55.0.rc0.799.gd6f94ed593-goog

[PATCH 2/6] mm/hugetlb: implement cache recycling and allocation

[Sourav Panda]

Implement the core recycling and allocation logic for the HugeTLB dynamic
cache.

Recycle surplus hugepages in free_huge_folio() up to max_cached_huge_pages.
Allocate from the cache in alloc_surplus_hugetlb_folio() if pages are
available.

To ensure safety and architecture correctness:
- Skip recycling of hardware-poisoned pages in free_huge_folio() and
  skip poisoned pages in the cache during allocation.
- Call arch_clear_hugetlb_flags() before caching a page to clear stale
  architecture-specific metadata (e.g. ARM64 MTE tags, dcache dirty).
- Use MRU (Most Recently Used) policy for allocation (taking from the
  tail of the cache list) to prefer hot pages.

Signed-off-by: Sourav Panda <souravpanda@google.com>
---
 include/linux/hugetlb.h |  4 +++
 mm/hugetlb.c            | 76 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 77 insertions(+), 3 deletions(-)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 4768f52ddd35..e882c99780b6 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -650,6 +650,10 @@ HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable)
 HPAGEFLAG(Cma, cma)
 #ifdef CONFIG_HUGETLB_CACHE
 HPAGEFLAG(Cached, cached)
+#else
+static inline bool folio_test_hugetlb_cached(const struct folio *folio) { return false; }
+static inline void folio_clear_hugetlb_cached(struct folio *folio) { }
+static inline void folio_set_hugetlb_cached(struct folio *folio) { }
 #endif
 
 #ifdef CONFIG_HUGETLB_PAGE
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 15be3cf54606..d00aa67b8e13 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -125,6 +125,11 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
 		unsigned long start, unsigned long end, bool take_locks);
 static struct resv_map *vma_resv_map(struct vm_area_struct *vma);
 
+#ifdef CONFIG_HUGETLB_CACHE
+static void hugetlb_cache_add(struct hstate *h, struct folio *folio, bool from_surplus);
+static void hugetlb_cache_remove(struct hstate *h, struct folio *folio, bool to_surplus);
+#endif
+
 static inline bool subpool_is_free(struct hugepage_subpool *spool)
 {
 	if (spool->count)
@@ -1796,6 +1801,15 @@ void free_huge_folio(struct folio *folio)
 		spin_unlock_irqrestore(&hugetlb_lock, flags);
 		update_and_free_hugetlb_folio(h, folio, true);
 	} else if (h->surplus_huge_pages_node[nid]) {
+#ifdef CONFIG_HUGETLB_CACHE
+		if (h->nr_cached_hugepages < h->max_cached_huge_pages &&
+		    !folio_test_hwpoison(folio)) {
+			arch_clear_hugetlb_flags(folio);
+			hugetlb_cache_add(h, folio, true);
+			spin_unlock_irqrestore(&hugetlb_lock, flags);
+			return;
+		}
+#endif
 		/* remove the page from active list */
 		remove_hugetlb_folio(h, folio, true);
 		spin_unlock_irqrestore(&hugetlb_lock, flags);
@@ -2035,8 +2049,14 @@ int dissolve_free_hugetlb_folio(struct folio *folio)
 		struct hstate *h = folio_hstate(folio);
 		bool adjust_surplus = false;
 
-		if (!available_huge_pages(h))
+		if (!available_huge_pages(h) && !folio_test_hugetlb_cached(folio))
+			goto out;
+
+		/* If the folio is currently isolated for page reporting, skip it */
+		if (folio_test_hugetlb_cached(folio) && list_empty(&folio->lru)) {
+			rc = -EBUSY;
 			goto out;
+		}
 
 		/*
 		 * We should make sure that the page is already on the free list
@@ -2128,6 +2148,50 @@ int dissolve_free_hugetlb_folios(unsigned long start_pfn, unsigned long end_pfn)
 	return rc;
 }
 
+#ifdef CONFIG_HUGETLB_CACHE
+static struct folio *get_cached_folio(struct hstate *h, int nid, nodemask_t *nmask)
+{
+	int node;
+	struct folio *folio;
+
+	if (h->nr_cached_hugepages == 0)
+		return NULL;
+
+	if (nid != NUMA_NO_NODE && (!nmask || node_isset(nid, *nmask))) {
+		list_for_each_entry_reverse(folio, &h->hugepage_cache_lists[nid], lru) {
+			if (!folio_test_hwpoison(folio))
+				goto found;
+		}
+	}
+
+	if (nmask) {
+		for_each_node_mask(node, *nmask) {
+			list_for_each_entry_reverse(folio, &h->hugepage_cache_lists[node], lru) {
+				if (!folio_test_hwpoison(folio))
+					goto found;
+			}
+		}
+	} else {
+		for_each_node_state(node, N_MEMORY) {
+			list_for_each_entry_reverse(folio, &h->hugepage_cache_lists[node], lru) {
+				if (!folio_test_hwpoison(folio))
+					goto found;
+			}
+		}
+	}
+	return NULL;
+
+found:
+	hugetlb_cache_remove(h, folio, true);
+	return folio;
+}
+#else
+static inline struct folio *get_cached_folio(struct hstate *h, int nid, nodemask_t *nmask)
+{
+	return NULL;
+}
+#endif
+
 /*
  * Allocates a fresh surplus page from the page allocator.
  */
@@ -2136,10 +2200,16 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
 {
 	struct folio *folio = NULL;
 
+	spin_lock_irq(&hugetlb_lock);
+	folio = get_cached_folio(h, nid, nmask);
+	if (folio) {
+		spin_unlock_irq(&hugetlb_lock);
+		return folio;
+	}
+
 	if (hstate_is_gigantic_no_runtime(h))
-		return NULL;
+		goto out_unlock;
 
-	spin_lock_irq(&hugetlb_lock);
 	if (h->surplus_huge_pages >= h->nr_overcommit_huge_pages)
 		goto out_unlock;
 	spin_unlock_irq(&hugetlb_lock);
-- 
2.55.0.rc0.799.gd6f94ed593-goog

[PATCH 3/6] mm/hugetlb: add sysfs interfaces for cache

[Sourav Panda]

Introduce sysfs interfaces to control and monitor the HugeTLB dynamic
cache.

Expose the following attributes under:
/sys/kernel/mm/hugepages/hugepages-<size>/
- max_cached_huge_pages: Max limit of cached hugepages (global).
- nr_cached_hugepages: Current number of cached hugepages. Writing to
  this file allows manual expansion or contraction of the cache.
  These interfaces support writing delta values (e.g. "+1" or "-1")
  in addition to absolute values, allowing orchestrators (e.g. borglet)
  to dynamically scale the cache without racing against the kernel
  shrinker.

Also expose node-specific versions under node sysfs if CONFIG_NUMA.

Implement adjust_cached_huge_pages to handle the actual expansion
(allocating fresh hugepages to cache) and contraction
(freeing cached hugepages to buddy).

For expansion and contraction correctness:
- Global expansion interleaves allocations across allowed nodes.
- Node-specific expansion enforces the global cache limit ceiling and
  overcommit limits, uses __GFP_THISNODE to prevent fallback, and accounts
  to the actual allocated node using folio_nid().
- Contraction uses LRU (Least Recently Used) policy (taking from the
  head of the cache list) to discard cold pages first.

Signed-off-by: Sourav Panda <souravpanda@google.com>
---
 mm/hugetlb.c          | 230 ++++++++++++++++++++++++++++++++++++++++++
 mm/hugetlb_internal.h |   9 ++
 mm/hugetlb_sysfs.c    | 158 +++++++++++++++++++++++++++++
 3 files changed, 397 insertions(+)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index d00aa67b8e13..f18a3123cbcb 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -4171,6 +4171,236 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 	return -EBUSY;
 }
 
+#ifdef CONFIG_HUGETLB_CACHE
+static int adjust_cached_huge_pages(struct hstate *h, long count, bool is_delta, int nid,
+				    nodemask_t *nodes_allowed, bool update_limit)
+{
+	struct folio *folio;
+	int err = 0;
+
+	mutex_lock(&h->resize_lock);
+
+	if (is_delta) {
+		long current_val;
+
+		if (nid == NUMA_NO_NODE)
+			current_val = update_limit ? h->max_cached_huge_pages :
+						     h->nr_cached_hugepages;
+		else
+			current_val = update_limit ? h->max_cached_huge_pages_node[nid] :
+						     h->nr_cached_hugepages_node[nid];
+
+		current_val += count;
+		if (current_val < 0)
+			current_val = 0;
+		count = current_val;
+	}
+
+	if (nid == NUMA_NO_NODE) {
+		int temp_next_node = first_node(*nodes_allowed);
+
+		if (update_limit) {
+			h->max_cached_huge_pages = count;
+		} else if (count > h->max_cached_huge_pages) {
+			err = -EINVAL;
+			goto out;
+		}
+
+		/* Expansion */
+		while (!update_limit) {
+			unsigned long curr;
+
+			spin_lock_irq(&hugetlb_lock);
+			curr = h->nr_cached_hugepages;
+			spin_unlock_irq(&hugetlb_lock);
+
+			if (curr >= count)
+				break;
+
+			folio = alloc_pool_huge_folio(h, nodes_allowed,
+						      NULL, &temp_next_node);
+			if (!folio) {
+				err = -ENOMEM;
+				break;
+			}
+
+			spin_lock_irq(&hugetlb_lock);
+			if (h->nr_cached_hugepages >= count) {
+				spin_unlock_irq(&hugetlb_lock);
+				update_and_free_hugetlb_folio(h, folio, false);
+				break;
+			}
+
+			account_new_hugetlb_folio(h, folio);
+
+			hugetlb_cache_add(h, folio, false);
+			spin_unlock_irq(&hugetlb_lock);
+			cond_resched();
+		}
+
+		/* Contraction */
+		while (1) {
+			unsigned long curr;
+			struct folio *folio = NULL;
+
+			spin_lock_irq(&hugetlb_lock);
+			curr = h->nr_cached_hugepages;
+			if (curr <= count) {
+				spin_unlock_irq(&hugetlb_lock);
+				break;
+			}
+
+			int node;
+
+			for_each_node_mask(node, *nodes_allowed) {
+				if (!list_empty(&h->hugepage_cache_lists[node])) {
+					folio = list_first_entry(&h->hugepage_cache_lists[node],
+								 struct folio, lru);
+					break;
+				}
+			}
+
+			if (!folio) {
+				spin_unlock_irq(&hugetlb_lock);
+				break;
+			}
+
+			remove_hugetlb_folio(h, folio, false);
+			spin_unlock_irq(&hugetlb_lock);
+
+			update_and_free_hugetlb_folio(h, folio, false);
+			cond_resched();
+		}
+	} else {
+		if (update_limit) {
+			h->max_cached_huge_pages_node[nid] = count;
+		} else if (count > h->max_cached_huge_pages_node[nid]) {
+			err = -EINVAL;
+			goto out;
+		}
+
+		/* Node expansion */
+		while (!update_limit) {
+			unsigned long curr;
+			gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE;
+
+			spin_lock_irq(&hugetlb_lock);
+			curr = h->nr_cached_hugepages_node[nid];
+			if (h->nr_cached_hugepages >= h->max_cached_huge_pages) {
+				spin_unlock_irq(&hugetlb_lock);
+				err = -ENOSPC;
+				break;
+			}
+			spin_unlock_irq(&hugetlb_lock);
+
+			if (curr >= count)
+				break;
+
+			folio = alloc_fresh_hugetlb_folio(h, gfp_mask,
+							  nid, nodes_allowed);
+			if (!folio) {
+				err = -ENOMEM;
+				break;
+			}
+
+			spin_lock_irq(&hugetlb_lock);
+			if (h->nr_cached_hugepages_node[nid] >= count) {
+				spin_unlock_irq(&hugetlb_lock);
+				update_and_free_hugetlb_folio(h, folio, false);
+				break;
+			}
+
+			account_new_hugetlb_folio(h, folio);
+
+			hugetlb_cache_add(h, folio, false);
+			spin_unlock_irq(&hugetlb_lock);
+			cond_resched();
+		}
+
+		/* Node contraction */
+		while (1) {
+			unsigned long curr;
+			struct folio *folio = NULL;
+
+			spin_lock_irq(&hugetlb_lock);
+			curr = h->nr_cached_hugepages_node[nid];
+			if (curr <= count) {
+				spin_unlock_irq(&hugetlb_lock);
+				break;
+			}
+
+			if (!list_empty(&h->hugepage_cache_lists[nid]))
+				folio = list_first_entry(&h->hugepage_cache_lists[nid],
+							 struct folio, lru);
+
+			if (!folio) {
+				spin_unlock_irq(&hugetlb_lock);
+				break;
+			}
+
+			remove_hugetlb_folio(h, folio, false);
+			spin_unlock_irq(&hugetlb_lock);
+
+			update_and_free_hugetlb_folio(h, folio, false);
+			cond_resched();
+		}
+	}
+
+out:
+	mutex_unlock(&h->resize_lock);
+	return err;
+}
+
+ssize_t __nr_cached_hugepages_store_common(bool obey_mempolicy,
+					   struct hstate *h, int nid,
+					   long count, bool is_delta, size_t len)
+{
+	int err;
+	nodemask_t nodes_allowed, *n_mask;
+
+	if (hstate_is_gigantic_no_runtime(h))
+		return -EINVAL;
+
+	if (nid == NUMA_NO_NODE) {
+		if (!(obey_mempolicy &&
+		      init_nodemask_of_mempolicy(&nodes_allowed)))
+			n_mask = &node_states[N_MEMORY];
+		else
+			n_mask = &nodes_allowed;
+	} else {
+		init_nodemask_of_node(&nodes_allowed, nid);
+		n_mask = &nodes_allowed;
+	}
+
+	err = adjust_cached_huge_pages(h, count, is_delta, nid, n_mask, false);
+
+	return err ? err : len;
+}
+
+ssize_t __max_cached_huge_pages_store_common(bool obey_mempolicy,
+					     struct hstate *h, int nid,
+					     long count, bool is_delta, size_t len)
+{
+	int err;
+	nodemask_t nodes_allowed, *n_mask;
+
+	if (hstate_is_gigantic_no_runtime(h))
+		return -EINVAL;
+
+	if (nid == NUMA_NO_NODE) {
+		if (!(obey_mempolicy &&
+		      init_nodemask_of_mempolicy(&nodes_allowed)))
+			n_mask = &node_states[N_MEMORY];
+		else
+			n_mask = &nodes_allowed;
+	} else {
+		init_nodemask_of_node(&nodes_allowed, nid);
+		n_mask = &nodes_allowed;
+	}
+
+	err = adjust_cached_huge_pages(h, count, is_delta, nid, n_mask, true);
+
+	return err ? err : len;
+}
+#endif /* CONFIG_HUGETLB_CACHE */
+
 ssize_t __nr_hugepages_store_common(bool obey_mempolicy,
 					   struct hstate *h, int nid,
 					   unsigned long count, size_t len)
diff --git a/mm/hugetlb_internal.h b/mm/hugetlb_internal.h
index 1d2f870deccf..16ed9ac9dc05 100644
--- a/mm/hugetlb_internal.h
+++ b/mm/hugetlb_internal.h
@@ -114,4 +114,13 @@ extern void hugetlb_sysctl_init(void);
 static inline void hugetlb_sysctl_init(void) { }
 #endif
 
+#ifdef CONFIG_HUGETLB_CACHE
+ssize_t __nr_cached_hugepages_store_common(bool obey_mempolicy,
+					   struct hstate *h, int nid,
+					   long count, bool is_delta, size_t len);
+ssize_t __max_cached_huge_pages_store_common(bool obey_mempolicy,
+					     struct hstate *h, int nid,
+					     long count, bool is_delta, size_t len);
+#endif
+
 #endif /* _LINUX_HUGETLB_INTERNAL_H */
diff --git a/mm/hugetlb_sysfs.c b/mm/hugetlb_sysfs.c
index 79ece91406bf..682a3805e6ff 100644
--- a/mm/hugetlb_sysfs.c
+++ b/mm/hugetlb_sysfs.c
@@ -277,6 +277,152 @@ static ssize_t demote_size_store(struct kobject *kobj,
 }
 HSTATE_ATTR(demote_size);
 
+#ifdef CONFIG_HUGETLB_CACHE
+static ssize_t nr_cached_hugepages_show_common(struct kobject *kobj,
+					       struct kobj_attribute *attr, char *buf)
+{
+	struct hstate *h;
+	unsigned long nr_cached;
+	int nid;
+
+	h = kobj_to_hstate(kobj, &nid);
+	if (nid == NUMA_NO_NODE)
+		nr_cached = h->nr_cached_hugepages;
+	else
+		nr_cached = h->nr_cached_hugepages_node[nid];
+
+	return sysfs_emit(buf, "%lu\n", nr_cached);
+}
+
+static ssize_t nr_cached_hugepages_store_common(bool obey_mempolicy,
+						struct kobject *kobj, const char *buf,
+						size_t len)
+{
+	struct hstate *h;
+	long count;
+	bool is_delta = false;
+	int nid;
+	int err;
+	const char *p = skip_spaces(buf);
+
+	if (*p == '+' || *p == '-') {
+		is_delta = true;
+		err = kstrtol(p, 10, &count);
+	} else {
+		unsigned long ucount;
+
+		err = kstrtoul(p, 10, &ucount);
+		count = (long)ucount;
+	}
+	if (err)
+		return err;
+
+	h = kobj_to_hstate(kobj, &nid);
+	return __nr_cached_hugepages_store_common(obey_mempolicy, h, nid, count, is_delta, len);
+}
+
+static ssize_t nr_cached_hugepages_show(struct kobject *kobj,
+					struct kobj_attribute *attr, char *buf)
+{
+	return nr_cached_hugepages_show_common(kobj, attr, buf);
+}
+
+static ssize_t nr_cached_hugepages_store(struct kobject *kobj,
+					 struct kobj_attribute *attr, const char *buf, size_t len)
+{
+	return nr_cached_hugepages_store_common(false, kobj, buf, len);
+}
+HSTATE_ATTR(nr_cached_hugepages);
+
+#ifdef CONFIG_NUMA
+static ssize_t nr_cached_hugepages_mempolicy_show(struct kobject *kobj,
+						  struct kobj_attribute *attr,
+						  char *buf)
+{
+	return nr_cached_hugepages_show_common(kobj, attr, buf);
+}
+
+static ssize_t nr_cached_hugepages_mempolicy_store(struct kobject *kobj,
+						   struct kobj_attribute *attr,
+						   const char *buf, size_t len)
+{
+	return nr_cached_hugepages_store_common(true, kobj, buf, len);
+}
+HSTATE_ATTR(nr_cached_hugepages_mempolicy);
+#endif
+
+static ssize_t max_cached_huge_pages_show_common(struct kobject *kobj,
+						 struct kobj_attribute *attr, char *buf)
+{
+	struct hstate *h;
+	unsigned long max_cached;
+	int nid;
+
+	h = kobj_to_hstate(kobj, &nid);
+	if (nid == NUMA_NO_NODE)
+		max_cached = h->max_cached_huge_pages;
+	else
+		max_cached = h->max_cached_huge_pages_node[nid];
+
+	return sysfs_emit(buf, "%lu\n", max_cached);
+}
+
+static ssize_t max_cached_huge_pages_store_common(bool obey_mempolicy,
+						  struct kobject *kobj, const char *buf,
+						  size_t len)
+{
+	struct hstate *h;
+	long count;
+	bool is_delta = false;
+	int nid;
+	int err;
+	const char *p = skip_spaces(buf);
+
+	if (*p == '+' || *p == '-') {
+		is_delta = true;
+		err = kstrtol(p, 10, &count);
+	} else {
+		unsigned long ucount;
+
+		err = kstrtoul(p, 10, &ucount);
+		count = (long)ucount;
+	}
+	if (err)
+		return err;
+
+	h = kobj_to_hstate(kobj, &nid);
+	return __max_cached_huge_pages_store_common(obey_mempolicy, h, nid, count, is_delta, len);
+}
+
+static ssize_t max_cached_huge_pages_show(struct kobject *kobj,
+					  struct kobj_attribute *attr, char *buf)
+{
+	return max_cached_huge_pages_show_common(kobj, attr, buf);
+}
+
+static ssize_t max_cached_huge_pages_store(struct kobject *kobj,
+					   struct kobj_attribute *attr, const char *buf, size_t len)
+{
+	return max_cached_huge_pages_store_common(false, kobj, buf, len);
+}
+HSTATE_ATTR(max_cached_huge_pages);
+
+#ifdef CONFIG_NUMA
+static ssize_t max_cached_huge_pages_mempolicy_show(struct kobject *kobj,
+						    struct kobj_attribute *attr,
+						    char *buf)
+{
+	return max_cached_huge_pages_show_common(kobj, attr, buf);
+}
+
+static ssize_t max_cached_huge_pages_mempolicy_store(struct kobject *kobj,
+						     struct kobj_attribute *attr,
+						     const char *buf, size_t len)
+{
+	return max_cached_huge_pages_store_common(true, kobj, buf, len);
+}
+HSTATE_ATTR(max_cached_huge_pages_mempolicy);
+#endif
+#endif /* CONFIG_HUGETLB_CACHE */
+
 static struct attribute *hstate_attrs[] = {
 	&nr_hugepages_attr.attr,
 	&nr_overcommit_hugepages_attr.attr,
@@ -285,6 +431,14 @@ static struct attribute *hstate_attrs[] = {
 	&surplus_hugepages_attr.attr,
 #ifdef CONFIG_NUMA
 	&nr_hugepages_mempolicy_attr.attr,
+#endif
+#ifdef CONFIG_HUGETLB_CACHE
+	&nr_cached_hugepages_attr.attr,
+	&max_cached_huge_pages_attr.attr,
+#ifdef CONFIG_NUMA
+	&nr_cached_hugepages_mempolicy_attr.attr,
+	&max_cached_huge_pages_mempolicy_attr.attr,
+#endif
 #endif
 	NULL,
 };
@@ -359,6 +513,10 @@ static struct attribute *per_node_hstate_attrs[] = {
 	&nr_hugepages_attr.attr,
 	&free_hugepages_attr.attr,
 	&surplus_hugepages_attr.attr,
+#ifdef CONFIG_HUGETLB_CACHE
+	&nr_cached_hugepages_attr.attr,
+	&max_cached_huge_pages_attr.attr,
+#endif
 	NULL,
 };
 
-- 
2.55.0.rc0.799.gd6f94ed593-goog

[PATCH 4/6] mm/hugetlb: add memory shrinker for cache

[Sourav Panda]

Register a memory shrinker for the HugeTLB dynamic cache to
reclaim cached pages under memory pressure.

Implement hugetlb_shrinker_count to report the number of cached pages
(currently limited to gigantic pages in the shrinker) and
hugetlb_shrinker_scan to free them back to the buddy allocator.

The shrinker scan uses LRU policy to reclaim the coldest pages first.

Signed-off-by: Sourav Panda <souravpanda@google.com>
---
 mm/hugetlb.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index f18a3123cbcb..1bc0198a695c 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -4399,6 +4399,68 @@ ssize_t __max_cached_huge_pages_store_common(bool obey_mempolicy,
 
 	return err ? err : len;
 }
+
+static unsigned long hugetlb_shrinker_count(struct shrinker *shrink,
+					    struct shrink_control *sc)
+{
+	struct hstate *h;
+	unsigned long count = 0;
+	int nid = sc->nid;
+
+	for_each_hstate(h) {
+		count += h->nr_cached_hugepages_node[nid];
+	}
+	return count;
+}
+
+static unsigned long hugetlb_shrinker_scan(struct shrinker *shrink,
+					   struct shrink_control *sc)
+{
+	struct hstate *h;
+	int nid = sc->nid;
+	unsigned long freed = 0;
+	unsigned long to_scan = sc->nr_to_scan;
+
+	for_each_hstate(h) {
+		while (freed < to_scan) {
+			struct folio *folio = NULL;
+			bool atomic = true;
+
+			/*
+			 * Only do synchronous vmemmap restoration if the caller
+			 * context allows blocking and FS reclaim (GFP_KERNEL compatible).
+			 * Otherwise, defer to workqueue to avoid lock inversion.
+			 */
+			if (gfpflags_allow_blocking(sc->gfp_mask) &&
+			    (sc->gfp_mask & __GFP_FS))
+				atomic = false;
+
+			spin_lock_irq(&hugetlb_lock);
+			if (list_empty(&h->hugepage_cache_lists[nid])) {
+				spin_unlock_irq(&hugetlb_lock);
+				break;
+			}
+
+			folio = list_first_entry(&h->hugepage_cache_lists[nid],
+						 struct folio, lru);
+
+			remove_hugetlb_folio(h, folio, false);
+			spin_unlock_irq(&hugetlb_lock);
+
+			update_and_free_hugetlb_folio(h, folio, atomic);
+
+			freed++;
+			if (current->reclaim_state)
+				current->reclaim_state->reclaimed += pages_per_huge_page(h);
+		}
+		if (freed >= to_scan)
+			break;
+	}
+
+	return freed;
+}
+
+static struct shrinker *hugetlb_cached_pages_shrinker;
 #endif /* CONFIG_HUGETLB_CACHE */
 
 ssize_t __nr_hugepages_store_common(bool obey_mempolicy,
@@ -4502,6 +4564,21 @@ static int __init hugetlb_init(void)
 
 	for (i = 0; i < num_fault_mutexes; i++)
 		mutex_init(&hugetlb_fault_mutex_table[i]);
+
+#ifdef CONFIG_HUGETLB_CACHE
+	hugetlb_cached_pages_shrinker = shrinker_alloc(SHRINKER_NUMA_AWARE |
+						       SHRINKER_NONSLAB,
+						       "hugetlb-cached-pages");
+	if (!hugetlb_cached_pages_shrinker) {
+		pr_warn("HugeTLB: Failed to allocate shrinker\n");
+	} else {
+		hugetlb_cached_pages_shrinker->count_objects = hugetlb_shrinker_count;
+		hugetlb_cached_pages_shrinker->scan_objects = hugetlb_shrinker_scan;
+		hugetlb_cached_pages_shrinker->batch = 1;
+		shrinker_register(hugetlb_cached_pages_shrinker);
+	}
+#endif
+
 	return 0;
 }
 subsys_initcall(hugetlb_init);
-- 
2.55.0.rc0.799.gd6f94ed593-goog

[PATCH 5/6] Documentation/admin-guide/mm/hugetlbpage.rst: document cache interfaces

[Sourav Panda]

Document the newly introduced sysfs interfaces for the HugeTLB
dynamic cache. This includes max_cached_huge_pages,
nr_cached_hugepages and their NUMA/mempolicy variants.

Signed-off-by: Sourav Panda <souravpanda@google.com>
---
 Documentation/admin-guide/mm/hugetlbpage.rst | 34 ++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst
index 67a941903fd2..6b96779e2584 100644
--- a/Documentation/admin-guide/mm/hugetlbpage.rst
+++ b/Documentation/admin-guide/mm/hugetlbpage.rst
@@ -264,6 +264,10 @@ pages may exist::
 	free_hugepages
 	resv_hugepages
 	surplus_hugepages
+	max_cached_huge_pages
+	max_cached_huge_pages_mempolicy
+	nr_cached_hugepages
+	nr_cached_hugepages_mempolicy
 
 The demote interfaces provide the ability to split a huge page into
 smaller huge pages.  For example, the x86 architecture supports both
@@ -286,8 +290,28 @@ demote
         actually demoted, compare the value of nr_hugepages before and after
         writing to the demote interface.  demote is a write only interface.
 
-The interfaces which are the same as in ``/proc`` (all except demote and
-demote_size) function as described above for the default huge page-sized case.
+max_cached_huge_pages
+        is the maximum number of cached hugepages for this size (global).
+        Surplus hugepages freed by applications are recycled into this cache
+        instead of being released back to the buddy allocator immediately.
+        Subsequent allocations for surplus pages will prefer this cache.
+        A user with root privileges can write to this file to set the limit.
+        This interface accepts absolute values as well as relative adjustments
+        using +1 or -1.
+
+max_cached_huge_pages_mempolicy
+        is the same as max_cached_huge_pages, but the nodes from which pages
+        are cached are controlled by the memory policy of the task.
+
+nr_cached_hugepages
+        reports the current number of cached hugepages. Writing to this file
+        allows manual expansion or contraction of the cache.
+        A user with root privileges can write to this file.
+        This interface accepts absolute values as well as relative adjustments
+        using +1 or -1.
+
+nr_cached_hugepages_mempolicy
+        is the same as nr_cached_hugepages, but obeys the memory policy.
+
+The interfaces which are the same as in ``/proc`` (all except demote,
+demote_size, and the cache interfaces) function as described above
+for the default huge page-sized case.
 
 .. _mem_policy_and_hp_alloc:
 
@@ -372,6 +396,8 @@ contains the following attribute files::
 	nr_hugepages
 	free_hugepages
 	surplus_hugepages
+	nr_cached_hugepages
+	max_cached_huge_pages
 
 The free\_' and surplus\_' attribute files are read-only.  They return the number
 of free and surplus [overcommitted] huge pages, respectively, on the parent
@@ -382,6 +408,10 @@ specified node.  When this attribute is written, the number of persistent huge
 pages on the parent node will be adjusted to the specified value, if sufficient
 resources exist, regardless of the task's mempolicy or cpuset constraints.
 
+The ``nr_cached_hugepages`` and ``max_cached_huge_pages`` attributes return the
+number of cached huge pages and the max limit on the specified node, respectively.
+Writing to these attributes adjusts the cache on the parent node.
+
 Note that the number of overcommit and reserve pages remain global quantities,
 as we don't know until fault time, when the faulting task's mempolicy is
 applied, from which node the huge page allocation will be attempted.
-- 
2.55.0.rc0.799.gd6f94ed593-goog

[PATCH 6/6] mm/hugetlb: support free page reporting for cached hugepages

[Sourav Panda]

Implement free page reporting for the HugeTLB dynamic cache.

Register HugeTLB hstates with the page reporting framework if they
support caching.

When pages are added to the cache (either via recycling in
free_huge_folio or direct population via sysfs nr_cached_hugepages),
trigger a page reporting cycle.

Page reporting isolates pages from the cache list, reports them to the
hypervisor via virtio-balloon, and then drains them back to the cache
list, marking them as reported (HPG_reported).

If a page is allocated from the cache, or reclaimed, clear the reported
flag.

Signed-off-by: Sourav Panda <souravpanda@google.com>
---
 include/linux/hugetlb.h        |  27 +++++++
 include/linux/page_reporting.h |   1 +
 mm/hugetlb.c                   | 125 +++++++++++++++++++++++++++++++++
 mm/page_reporting.c            |  10 ++-
 mm/page_reporting.h            |   6 ++
 5 files changed, 163 insertions(+), 6 deletions(-)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index e882c99780b6..bde23edcb803 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -587,6 +587,7 @@ enum hugetlb_page_flags {
 	HPG_cma,
 #ifdef CONFIG_HUGETLB_CACHE
 	HPG_cached,
+	HPG_reported,
 #endif
 	__NR_HPAGEFLAGS,
 };
@@ -650,10 +651,14 @@ HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable)
 HPAGEFLAG(Cma, cma)
 #ifdef CONFIG_HUGETLB_CACHE
 HPAGEFLAG(Cached, cached)
+HPAGEFLAG(Reported, reported)
 #else
 static inline bool folio_test_hugetlb_cached(const struct folio *folio) { return false; }
 static inline void folio_clear_hugetlb_cached(struct folio *folio) { }
 static inline void folio_set_hugetlb_cached(struct folio *folio) { }
+static inline bool folio_test_hugetlb_reported(const struct folio *folio) { return false; }
+static inline void folio_clear_hugetlb_reported(struct folio *folio) { }
+static inline void folio_set_hugetlb_reported(struct folio *folio) { }
 #endif
 
 #ifdef CONFIG_HUGETLB_PAGE
@@ -1093,6 +1098,20 @@ void hugetlb_unregister_node(struct node *node);
  */
 bool is_raw_hwpoison_page_in_hugepage(struct page *page);
 
+#ifdef CONFIG_HUGETLB_CACHE
+struct page_reporting_dev_info;
+int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
+			   struct scatterlist *sgl);
+#else /* !CONFIG_HUGETLB_CACHE */
+struct page_reporting_dev_info;
+struct scatterlist;
+static inline int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
+					 struct scatterlist *sgl)
+{
+	return 0;
+}
+#endif /* CONFIG_HUGETLB_CACHE */
+
 static inline unsigned long huge_page_mask_align(struct file *file)
 {
 	return PAGE_MASK & ~huge_page_mask(hstate_file(file));
@@ -1311,6 +1330,14 @@ static inline bool hugetlbfs_pagecache_present(
 static inline void hugetlb_bootmem_alloc(void)
 {
 }
+
+struct page_reporting_dev_info;
+struct scatterlist;
+static inline int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
+					 struct scatterlist *sgl)
+{
+	return 0;
+}
 #endif	/* CONFIG_HUGETLB_PAGE */
 
 static inline spinlock_t *huge_pte_lock(struct hstate *h,
diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
index 9d4ca5c218a0..bd5a5a293dc3 100644
--- a/include/linux/page_reporting.h
+++ b/include/linux/page_reporting.h
@@ -27,4 +27,5 @@ struct page_reporting_dev_info {
 /* Tear-down and bring-up for page reporting devices */
 void page_reporting_unregister(struct page_reporting_dev_info *prdev);
 int page_reporting_register(struct page_reporting_dev_info *prdev);
+extern unsigned int page_reporting_order;
 #endif /*_LINUX_PAGE_REPORTING_H */
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 1bc0198a695c..110c566efc3b 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -46,7 +46,9 @@
 #include <linux/io.h>
 #include <linux/node.h>
 #include <linux/page_owner.h>
+#include <linux/page_reporting.h>
 #include "internal.h"
+#include "page_reporting.h"
 #include "hugetlb_vmemmap.h"
 #include "hugetlb_cma.h"
 #include "hugetlb_internal.h"
@@ -1414,6 +1416,7 @@ static void hugetlb_cache_remove(struct hstate *h, struct folio *folio, bool to_
 	list_del_init(&folio->lru);
 	folio_clear_hugetlb_freed(folio);
 	folio_clear_hugetlb_cached(folio);
+	folio_clear_hugetlb_reported(folio);
 	h->nr_cached_hugepages--;
 	h->nr_cached_hugepages_node[nid]--;
 	if (to_surplus) {
@@ -1806,6 +1809,7 @@ void free_huge_folio(struct folio *folio)
 		    !folio_test_hwpoison(folio)) {
 			arch_clear_hugetlb_flags(folio);
 			hugetlb_cache_add(h, folio, true);
+			page_reporting_notify_free(h->order);
 			spin_unlock_irqrestore(&hugetlb_lock, flags);
 			return;
 		}
@@ -4231,6 +4235,7 @@ static int adjust_cached_huge_pages(struct hstate *h, long count, bool is_delta,
 			account_new_hugetlb_folio(h, folio);
 
 			hugetlb_cache_add(h, folio, false);
+			page_reporting_notify_free(h->order);
 			spin_unlock_irq(&hugetlb_lock);
 			cond_resched();
 		}
@@ -4310,6 +4315,7 @@ static int adjust_cached_huge_pages(struct hstate *h, long count, bool is_delta,
 			account_new_hugetlb_folio(h, folio);
 
 			hugetlb_cache_add(h, folio, false);
+			page_reporting_notify_free(h->order);
 			spin_unlock_irq(&hugetlb_lock);
 			cond_resched();
 		}
@@ -7763,3 +7769,122 @@ void fixup_hugetlb_reservations(struct vm_area_struct *vma)
 	if (is_vm_hugetlb_page(vma))
 		clear_vma_resv_huge_pages(vma);
 }
+
+#ifdef CONFIG_HUGETLB_CACHE
+static void page_reporting_drain_hugetlb(struct hstate *h,
+					 struct scatterlist *sgl, unsigned int nents, bool reported)
+{
+	struct scatterlist *sg = sgl;
+	unsigned int left = nents;
+
+	spin_lock_irq(&hugetlb_lock);
+	do {
+		struct page *page = sg_page(sg);
+		struct folio *folio = page_folio(page);
+
+		hugetlb_cache_add(h, folio, false);
+
+		if (reported)
+			folio_set_hugetlb_reported(folio);
+	} while (--left && (sg = sg_next(sg)));
+	spin_unlock_irq(&hugetlb_lock);
+
+	sg_init_table(sgl, nents);
+}
+
+static int hugetlb_page_reporting_cycle(struct page_reporting_dev_info *prdev,
+					struct hstate *h, int nid,
+					struct scatterlist *sgl, unsigned int *offset)
+{
+	struct list_head *list = &h->hugepage_cache_lists[nid];
+	unsigned int page_len = huge_page_size(h);
+	struct folio *folio, *next;
+	long budget = 100; /* Limit */
+	int err = 0;
+
+	if (list_empty(list))
+		return err;
+
+	spin_lock_irq(&hugetlb_lock);
+
+	list_for_each_entry_safe(folio, next, list, lru) {
+		if (folio_test_hugetlb_reported(folio))
+			continue;
+
+		if (budget < 0) {
+			atomic_set(&prdev->state, PAGE_REPORTING_REQUESTED);
+			break;
+		}
+
+		if (*offset) {
+			/* Isolate */
+			hugetlb_cache_remove(h, folio, false);
+
+			--(*offset);
+			sg_set_page(&sgl[*offset], &folio->page, page_len, 0);
+			continue;
+		}
+
+		spin_unlock_irq(&hugetlb_lock);
+
+		err = prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY);
+		if (!err) {
+			pr_info("HugeTLB page reporting: reported %u pages of size %luKB\n",
+				PAGE_REPORTING_CAPACITY, huge_page_size(h) / 1024);
+		}
+
+		*offset = PAGE_REPORTING_CAPACITY;
+		budget--;
+
+		page_reporting_drain_hugetlb(h, sgl, PAGE_REPORTING_CAPACITY, !err);
+
+		spin_lock_irq(&hugetlb_lock);
+
+		if (list_empty(list))
+			break;
+		next = list_first_entry(list, struct folio, lru);
+
+		if (err)
+			break;
+	}
+
+	spin_unlock_irq(&hugetlb_lock);
+	return err;
+}
+
+int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
+			   struct scatterlist *sgl)
+{
+	struct hstate *h;
+	int err = 0;
+	int nid;
+
+	for_each_hstate(h) {
+		if (h->order < page_reporting_order)
+			continue;
+
+		for_each_online_node(nid) {
+			unsigned int offset = PAGE_REPORTING_CAPACITY;
+
+			err = hugetlb_page_reporting_cycle(prdev, h, nid, sgl, &offset);
+			if (err)
+				return err;
+
+			/* Report leftovers */
+			unsigned int leftover = PAGE_REPORTING_CAPACITY - offset;
+
+			if (leftover) {
+				struct scatterlist *sg = &sgl[offset];
+
+				err = prdev->report(prdev, sg, leftover);
+				page_reporting_drain_hugetlb(h, sg, leftover, !err);
+				if (err)
+					return err;
+				pr_info("HugeTLB page reporting: reported %u pages of size %luKB\n",
+					leftover, huge_page_size(h) / 1024);
+			}
+		}
+	}
+	return 0;
+}
+#endif
diff --git a/mm/page_reporting.c b/mm/page_reporting.c
index 7418f2e500bb..882496b8b2c9 100644
--- a/mm/page_reporting.c
+++ b/mm/page_reporting.c
@@ -2,6 +2,7 @@
 #include <linux/mm.h>
 #include <linux/mmzone.h>
 #include <linux/page_reporting.h>
+#include <linux/hugetlb.h>
 #include <linux/gfp.h>
 #include <linux/export.h>
 #include <linux/module.h>
@@ -50,12 +51,6 @@ EXPORT_SYMBOL_GPL(page_reporting_order);
 #define PAGE_REPORTING_DELAY	(2 * HZ)
 static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly;
 
-enum {
-	PAGE_REPORTING_IDLE = 0,
-	PAGE_REPORTING_REQUESTED,
-	PAGE_REPORTING_ACTIVE
-};
-
 /* request page reporting */
 static void
 __page_reporting_request(struct page_reporting_dev_info *prdev)
@@ -334,6 +329,9 @@ static void page_reporting_process(struct work_struct *work)
 			break;
 	}
 
+	if (!err)
+		err = hugetlb_page_reporting(prdev, sgl);
+
 	kfree(sgl);
 err_out:
 	/*
diff --git a/mm/page_reporting.h b/mm/page_reporting.h
index c51dbc228b94..74e05af4bac1 100644
--- a/mm/page_reporting.h
+++ b/mm/page_reporting.h
@@ -11,6 +11,12 @@
 #include <linux/scatterlist.h>
 
 #ifdef CONFIG_PAGE_REPORTING
+enum {
+	PAGE_REPORTING_IDLE = 0,
+	PAGE_REPORTING_REQUESTED,
+	PAGE_REPORTING_ACTIVE
+};
+
 DECLARE_STATIC_KEY_FALSE(page_reporting_enabled);
 extern unsigned int page_reporting_order;
 void __page_reporting_notify(void);
-- 
2.55.0.rc0.799.gd6f94ed593-goog

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Michael S. Tsirkin]

On Tue, Jul 07, 2026 at 06:42:29AM +0000, Sourav Panda wrote:
> Overview
> This patch series introduces a dynamic, NUMA-aware HugePage Cache,
> backed by a kernel shrinker to safely return memory under pressure, and
> integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
> specifically targeting gigantic (1GB) hugepages. The goal is to solve
> the tradeoff between allocation latency and memory
> fungibility in virtualized and heterogeneous cloud environments.
> 
> ---
> 
> The Core Problem: Allocation Latency vs. Memory Fungibility
> 
> With highly heterogeneous workloads, latency-critical applications demand
> gigantic hugepages. However, dynamic runtime allocation of 1GB pages
> from the buddy allocator (via CMA) is slow.
> 
> To bypass this latency, operators often pre-allocate hugepages
> statically. However, this locks up the memory: when the HugeTLB
> workloads are idle, that memory is completely unavailable for other
> buddy-allocator workloads (e.g., page cache, anonymous memory). If buddy
> memory is exhausted, the system will OOM even if gigabytes of HugeTLB
> pages are sitting idle.
> 
> This series resolves this challenge by delivering Dynamic Fungibility:
> 
> 1.  Dynamic Caching: Intercepts freed surplus hugepages and recycles
>     them into a NUMA-aware cache instead of dissolving them immediately.
> 2.  Fast Allocations: Satisfies subsequent dynamic allocations
>     instantly from this warm, local hugepage cache.
> 3.  Kernel Shrinker Integration: Registers a NUMA-aware kernel shrinker
>     to dynamically dissolve cached pages back to the buddy allocator
>     under memory pressure, restoring host/guest memory fungibility.
> 4.  Free Page Reporting Integration: For virtualized environments (Guest
>     VMs),

I was going to look into this part, thanks for working on this.

> cached pages trigger background Free Page Reporting via
>     virtio-balloon. This allows the host to reclaim the physical memory
>     while the guest retains its Vmemmap Optimization (HVO) metadata
>     savings (~14GB saved per 1TB VM)!

Why "!" - that's 1.5%, seems surprisingly modest.
Are you sure it's working as intended?

>   +---------------+   Slow Allocate  +---------------------+
>   | Buddy         | ---------------> | Active HugeTLB Page |
>   | Allocator     |                  +---------------------+
>   +---------------+                    |                ^
>     ^                                  | (1) Free to    | (2) Fast
>     | (3) Under                        v     Cache      |     Allocate
>     | Pressure                       +---------------------+
>     | Shrink()  <------------------- | HugeTLB Cache       |
>                                      +---------------------+
>                                        |
>                                        v (4) Free Page Reporting
>                                          (Host Reclaim in the case
>                                           of virtualization)
> 
> ---
> 
> Patch Series Structure
> 
> Patch 1/6: mm/hugetlb: add Kconfig and basic cache infrastructure
>   - Introduces the CONFIG_HUGETLB_CACHE option, hstate tracking fields,
>     and the HPG_cached page flag.
>   - Establishes the clean helper API (hugetlb_folio_is_cached(),
>     hugetlb_cache_remove(), hugetlb_cache_add()) and updates
>     remove_hugetlb_folio() to be cache-aware, eliminating inline #ifdef
>     blocks.
> Patch 2/6: mm/hugetlb: implement cache recycling and allocation
>   - Hooks up recycling in free_huge_folio() (up to the cache limit) and
>     allocation in alloc_surplus_hugetlb_folio().
>   - Implements MRU allocation policy for maximum warmth, poison safety
>     checks, and MTE/dcache cleaning.
> Patch 3/6: mm/hugetlb: add sysfs interfaces for cache
>   - Exposes global and per-node sysfs attributes (max_cached_huge_pages,
>     nr_cached_hugepages) for dynamic userspace control, including NUMA
>     memory policy scaling.
>   - Supports dynamic delta adjustments (+1/-1) to safely scale cache
>     sizes alongside concurrent background reclaim operations.
> Patch 4/6: mm/hugetlb: add memory shrinker for cache
>   - Registers a NUMA-aware kernel shrinker to evict and dissolve cached
>     gigantic pages back to buddy under memory pressure.
> Patch 5/6: Documentation/admin-guide/mm/hugetlbpage.rst: document cache
>     interfaces
>   - Documents the Kconfig option, sysfs attributes, and shrinker
>     behavior in the admin guide.
> Patch 6/6: mm/hugetlb: support free page reporting for cached hugepages
>   - Integrates the cache with the Free Page Reporting framework (virtio-
>     balloon), introducing the HPG_reported flag and the
>     reporting/isolation/draining lifecycle.
> Sourav Panda (6):
>   mm/hugetlb: add Kconfig and basic cache infrastructure
>   mm/hugetlb: implement cache recycling and allocation
>   mm/hugetlb: add sysfs interfaces for cache
>   mm/hugetlb: add memory shrinker for cache
>   Documentation/admin-guide/mm/hugetlbpage.rst: document cache
>     interfaces
>   mm/hugetlb: support free page reporting for cached hugepages
> 
>  Documentation/admin-guide/mm/hugetlbpage.rst |  34 +-
>  fs/Kconfig                                   |   9 +
>  include/linux/hugetlb.h                      |  45 ++
>  include/linux/page_reporting.h               |   1 +
>  mm/hugetlb.c                                 | 590 ++++++++++++++++++-
>  mm/hugetlb_internal.h                        |   9 +
>  mm/hugetlb_sysfs.c                           | 158 +++++
>  mm/page_reporting.c                          |  10 +-
>  mm/page_reporting.h                          |   6 +
>  9 files changed, 844 insertions(+), 18 deletions(-)
> 
> -- 
> 2.55.0.rc0.799.gd6f94ed593-goog

Re: [PATCH 6/6] mm/hugetlb: support free page reporting for cached hugepages

[Michael S. Tsirkin]

Thanks for the patch! Yet something to improve:

On Tue, Jul 07, 2026 at 06:42:35AM +0000, Sourav Panda wrote:
> Implement free page reporting for the HugeTLB dynamic cache.
> 
> Register HugeTLB hstates with the page reporting framework if they
> support caching.
> 
> When pages are added to the cache (either via recycling in
> free_huge_folio or direct population via sysfs nr_cached_hugepages),
> trigger a page reporting cycle.
> 
> Page reporting isolates pages from the cache list, reports them to the
> hypervisor via virtio-balloon, and then drains them back to the cache
> list, marking them as reported (HPG_reported).
> 
> If a page is allocated from the cache, or reclaimed, clear the reported
> flag.
> 
> Signed-off-by: Sourav Panda <souravpanda@google.com>
> ---
>  include/linux/hugetlb.h        |  27 +++++++
>  include/linux/page_reporting.h |   1 +
>  mm/hugetlb.c                   | 125 +++++++++++++++++++++++++++++++++
>  mm/page_reporting.c            |  10 ++-
>  mm/page_reporting.h            |   6 ++
>  5 files changed, 163 insertions(+), 6 deletions(-)
> 
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index e882c99780b6..bde23edcb803 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -587,6 +587,7 @@ enum hugetlb_page_flags {
>  	HPG_cma,
>  #ifdef CONFIG_HUGETLB_CACHE
>  	HPG_cached,
> +	HPG_reported,
>  #endif
>  	__NR_HPAGEFLAGS,
>  };
> @@ -650,10 +651,14 @@ HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable)
>  HPAGEFLAG(Cma, cma)
>  #ifdef CONFIG_HUGETLB_CACHE
>  HPAGEFLAG(Cached, cached)
> +HPAGEFLAG(Reported, reported)
>  #else
>  static inline bool folio_test_hugetlb_cached(const struct folio *folio) { return false; }
>  static inline void folio_clear_hugetlb_cached(struct folio *folio) { }
>  static inline void folio_set_hugetlb_cached(struct folio *folio) { }
> +static inline bool folio_test_hugetlb_reported(const struct folio *folio) { return false; }
> +static inline void folio_clear_hugetlb_reported(struct folio *folio) { }
> +static inline void folio_set_hugetlb_reported(struct folio *folio) { }
>  #endif
>  
>  #ifdef CONFIG_HUGETLB_PAGE
> @@ -1093,6 +1098,20 @@ void hugetlb_unregister_node(struct node *node);
>   */
>  bool is_raw_hwpoison_page_in_hugepage(struct page *page);
>  
> +#ifdef CONFIG_HUGETLB_CACHE
> +struct page_reporting_dev_info;
> +int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
> +			   struct scatterlist *sgl);
> +#else /* !CONFIG_HUGETLB_CACHE */
> +struct page_reporting_dev_info;
> +struct scatterlist;
> +static inline int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
> +					 struct scatterlist *sgl)
> +{
> +	return 0;
> +}
> +#endif /* CONFIG_HUGETLB_CACHE */
> +
>  static inline unsigned long huge_page_mask_align(struct file *file)
>  {
>  	return PAGE_MASK & ~huge_page_mask(hstate_file(file));
> @@ -1311,6 +1330,14 @@ static inline bool hugetlbfs_pagecache_present(
>  static inline void hugetlb_bootmem_alloc(void)
>  {
>  }
> +
> +struct page_reporting_dev_info;
> +struct scatterlist;
> +static inline int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
> +					 struct scatterlist *sgl)
> +{
> +	return 0;
> +}
>  #endif	/* CONFIG_HUGETLB_PAGE */
>  

why are there twice the stubs? And should this not depend on CONFIG_PAGE_REPORTING too?


>  static inline spinlock_t *huge_pte_lock(struct hstate *h,
> diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h
> index 9d4ca5c218a0..bd5a5a293dc3 100644
> --- a/include/linux/page_reporting.h
> +++ b/include/linux/page_reporting.h
> @@ -27,4 +27,5 @@ struct page_reporting_dev_info {
>  /* Tear-down and bring-up for page reporting devices */
>  void page_reporting_unregister(struct page_reporting_dev_info *prdev);
>  int page_reporting_register(struct page_reporting_dev_info *prdev);
> +extern unsigned int page_reporting_order;

again why twice.


>  #endif /*_LINUX_PAGE_REPORTING_H */
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 1bc0198a695c..110c566efc3b 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -46,7 +46,9 @@
>  #include <linux/io.h>
>  #include <linux/node.h>
>  #include <linux/page_owner.h>
> +#include <linux/page_reporting.h>
>  #include "internal.h"
> +#include "page_reporting.h"
>  #include "hugetlb_vmemmap.h"
>  #include "hugetlb_cma.h"
>  #include "hugetlb_internal.h"
> @@ -1414,6 +1416,7 @@ static void hugetlb_cache_remove(struct hstate *h, struct folio *folio, bool to_
>  	list_del_init(&folio->lru);
>  	folio_clear_hugetlb_freed(folio);
>  	folio_clear_hugetlb_cached(folio);
> +	folio_clear_hugetlb_reported(folio);
>  	h->nr_cached_hugepages--;
>  	h->nr_cached_hugepages_node[nid]--;
>  	if (to_surplus) {
> @@ -1806,6 +1809,7 @@ void free_huge_folio(struct folio *folio)
>  		    !folio_test_hwpoison(folio)) {
>  			arch_clear_hugetlb_flags(folio);
>  			hugetlb_cache_add(h, folio, true);
> +			page_reporting_notify_free(h->order);
>  			spin_unlock_irqrestore(&hugetlb_lock, flags);
>  			return;
>  		}
> @@ -4231,6 +4235,7 @@ static int adjust_cached_huge_pages(struct hstate *h, long count, bool is_delta,
>  			account_new_hugetlb_folio(h, folio);
>  
>  			hugetlb_cache_add(h, folio, false);
> +			page_reporting_notify_free(h->order);
>  			spin_unlock_irq(&hugetlb_lock);
>  			cond_resched();
>  		}
> @@ -4310,6 +4315,7 @@ static int adjust_cached_huge_pages(struct hstate *h, long count, bool is_delta,
>  			account_new_hugetlb_folio(h, folio);
>  
>  			hugetlb_cache_add(h, folio, false);
> +			page_reporting_notify_free(h->order);
>  			spin_unlock_irq(&hugetlb_lock);
>  			cond_resched();
>  		}
> @@ -7763,3 +7769,122 @@ void fixup_hugetlb_reservations(struct vm_area_struct *vma)
>  	if (is_vm_hugetlb_page(vma))
>  		clear_vma_resv_huge_pages(vma);
>  }
> +
> +#ifdef CONFIG_HUGETLB_CACHE
> +static void page_reporting_drain_hugetlb(struct hstate *h,
> +					 struct scatterlist *sgl, unsigned int nents, bool reported)
> +{
> +	struct scatterlist *sg = sgl;
> +	unsigned int left = nents;
> +
> +	spin_lock_irq(&hugetlb_lock);
> +	do {
> +		struct page *page = sg_page(sg);
> +		struct folio *folio = page_folio(page);
> +
> +		hugetlb_cache_add(h, folio, false);
> +
> +		if (reported)
> +			folio_set_hugetlb_reported(folio);
> +	} while (--left && (sg = sg_next(sg)));
> +	spin_unlock_irq(&hugetlb_lock);
> +
> +	sg_init_table(sgl, nents);
> +}
> +
> +static int hugetlb_page_reporting_cycle(struct page_reporting_dev_info *prdev,
> +					struct hstate *h, int nid,
> +					struct scatterlist *sgl, unsigned int *offset)
> +{
> +	struct list_head *list = &h->hugepage_cache_lists[nid];
> +	unsigned int page_len = huge_page_size(h);
> +	struct folio *folio, *next;
> +	long budget = 100; /* Limit */

why 100? and "Limit"? What kind of comment is this?


> +	int err = 0;
> +
> +	if (list_empty(list))
> +		return err;
> +
> +	spin_lock_irq(&hugetlb_lock);
> +
> +	list_for_each_entry_safe(folio, next, list, lru) {
> +		if (folio_test_hugetlb_reported(folio))
> +			continue;
> +
> +		if (budget < 0) {
> +			atomic_set(&prdev->state, PAGE_REPORTING_REQUESTED);
> +			break;
> +		}
> +
> +		if (*offset) {
> +			/* Isolate */
> +			hugetlb_cache_remove(h, folio, false);

are you sure this is right? in patch 2 it is:
       if (folio_test_hugetlb_cached(folio) && list_empty(&folio->lru))
           return -EBUSY;
and hugetlb_cache_remove clears HPG_cached.


> +
> +			--(*offset);
> +			sg_set_page(&sgl[*offset], &folio->page, page_len, 0);
> +			continue;
> +		}
> +
> +		spin_unlock_irq(&hugetlb_lock);
> +
> +		err = prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY);
> +		if (!err) {
> +			pr_info("HugeTLB page reporting: reported %u pages of size %luKB\n",
> +				PAGE_REPORTING_CAPACITY, huge_page_size(h) / 1024);

seriously?

> +		}
> +
> +		*offset = PAGE_REPORTING_CAPACITY;
> +		budget--;
> +
> +		page_reporting_drain_hugetlb(h, sgl, PAGE_REPORTING_CAPACITY, !err);
> +
> +		spin_lock_irq(&hugetlb_lock);
> +
> +		if (list_empty(list))
> +			break;
> +		next = list_first_entry(list, struct folio, lru);
> +
> +		if (err)
> +			break;
> +	}
> +
> +	spin_unlock_irq(&hugetlb_lock);
> +	return err;
> +}
> +
> +int hugetlb_page_reporting(struct page_reporting_dev_info *prdev,
> +			   struct scatterlist *sgl)
> +{
> +	struct hstate *h;
> +	int err = 0;
> +	int nid;
> +
> +	for_each_hstate(h) {
> +		if (h->order < page_reporting_order)
> +			continue;
> +
> +		for_each_online_node(nid) {
> +			unsigned int offset = PAGE_REPORTING_CAPACITY;
> +
> +			err = hugetlb_page_reporting_cycle(prdev, h, nid, sgl, &offset);
> +			if (err)
> +				return err;
> +
> +			/* Report leftovers */
> +			unsigned int leftover = PAGE_REPORTING_CAPACITY - offset;
> +
> +			if (leftover) {
> +				struct scatterlist *sg = &sgl[offset];
> +
> +				err = prdev->report(prdev, sg, leftover);
> +				page_reporting_drain_hugetlb(h, sg, leftover, !err);
> +				if (err)
> +					return err;
> +				pr_info("HugeTLB page reporting: reported %u pages of size %luKB\n",
> +					leftover, huge_page_size(h) / 1024);

again

> +			}
> +		}
> +	}
> +	return 0;
> +}
> +#endif
> diff --git a/mm/page_reporting.c b/mm/page_reporting.c
> index 7418f2e500bb..882496b8b2c9 100644
> --- a/mm/page_reporting.c
> +++ b/mm/page_reporting.c
> @@ -2,6 +2,7 @@
>  #include <linux/mm.h>
>  #include <linux/mmzone.h>
>  #include <linux/page_reporting.h>
> +#include <linux/hugetlb.h>
>  #include <linux/gfp.h>
>  #include <linux/export.h>
>  #include <linux/module.h>
> @@ -50,12 +51,6 @@ EXPORT_SYMBOL_GPL(page_reporting_order);
>  #define PAGE_REPORTING_DELAY	(2 * HZ)
>  static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly;
>  
> -enum {
> -	PAGE_REPORTING_IDLE = 0,
> -	PAGE_REPORTING_REQUESTED,
> -	PAGE_REPORTING_ACTIVE
> -};
> -
>  /* request page reporting */
>  static void
>  __page_reporting_request(struct page_reporting_dev_info *prdev)
> @@ -334,6 +329,9 @@ static void page_reporting_process(struct work_struct *work)
>  			break;
>  	}
>  
> +	if (!err)
> +		err = hugetlb_page_reporting(prdev, sgl);
> +
>  	kfree(sgl);
>  err_out:
>  	/*
> diff --git a/mm/page_reporting.h b/mm/page_reporting.h
> index c51dbc228b94..74e05af4bac1 100644
> --- a/mm/page_reporting.h
> +++ b/mm/page_reporting.h
> @@ -11,6 +11,12 @@
>  #include <linux/scatterlist.h>
>  
>  #ifdef CONFIG_PAGE_REPORTING
> +enum {
> +	PAGE_REPORTING_IDLE = 0,
> +	PAGE_REPORTING_REQUESTED,
> +	PAGE_REPORTING_ACTIVE
> +};
> +
>  DECLARE_STATIC_KEY_FALSE(page_reporting_enabled);
>  extern unsigned int page_reporting_order;
>  void __page_reporting_notify(void);
> -- 
> 2.55.0.rc0.799.gd6f94ed593-goog

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Sourav Panda]

On Mon, Jul 6, 2026 at 11:57 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Tue, Jul 07, 2026 at 06:42:29AM +0000, Sourav Panda wrote:
> > Overview
> > This patch series introduces a dynamic, NUMA-aware HugePage Cache,
> > backed by a kernel shrinker to safely return memory under pressure, and
> > integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
> > specifically targeting gigantic (1GB) hugepages. The goal is to solve
> > the tradeoff between allocation latency and memory
> > fungibility in virtualized and heterogeneous cloud environments.
> >
> > ---
> >
> > The Core Problem: Allocation Latency vs. Memory Fungibility
> >
> > With highly heterogeneous workloads, latency-critical applications demand
> > gigantic hugepages. However, dynamic runtime allocation of 1GB pages
> > from the buddy allocator (via CMA) is slow.
> >
> > To bypass this latency, operators often pre-allocate hugepages
> > statically. However, this locks up the memory: when the HugeTLB
> > workloads are idle, that memory is completely unavailable for other
> > buddy-allocator workloads (e.g., page cache, anonymous memory). If buddy
> > memory is exhausted, the system will OOM even if gigabytes of HugeTLB
> > pages are sitting idle.
> >
> > This series resolves this challenge by delivering Dynamic Fungibility:
> >
> > 1.  Dynamic Caching: Intercepts freed surplus hugepages and recycles
> >     them into a NUMA-aware cache instead of dissolving them immediately.
> > 2.  Fast Allocations: Satisfies subsequent dynamic allocations
> >     instantly from this warm, local hugepage cache.
> > 3.  Kernel Shrinker Integration: Registers a NUMA-aware kernel shrinker
> >     to dynamically dissolve cached pages back to the buddy allocator
> >     under memory pressure, restoring host/guest memory fungibility.
> > 4.  Free Page Reporting Integration: For virtualized environments (Guest
> >     VMs),
>
> I was going to look into this part, thanks for working on this.
>
> > cached pages trigger background Free Page Reporting via
> >     virtio-balloon. This allows the host to reclaim the physical memory
> >     while the guest retains its Vmemmap Optimization (HVO) metadata
> >     savings (~14GB saved per 1TB VM)!
>
> Why "!" - that's 1.5%, seems surprisingly modest.
> Are you sure it's working as intended?

I phrased it poorly, my bad! Thanks for pointing out Michael :)

That ~14GB is only the guest struct page savings. Meanwhile, the host
is reclaiming the full underlying physical memory of the cached
hugepages via FPR. I do not have a number for this yet.

Let me reword this in the next iteration.


>
> >   +---------------+   Slow Allocate  +---------------------+
> >   | Buddy         | ---------------> | Active HugeTLB Page |
> >   | Allocator     |                  +---------------------+
> >   +---------------+                    |                ^
> >     ^                                  | (1) Free to    | (2) Fast
> >     | (3) Under                        v     Cache      |     Allocate
> >     | Pressure                       +---------------------+
> >     | Shrink()  <------------------- | HugeTLB Cache       |
> >                                      +---------------------+
> >                                        |
> >                                        v (4) Free Page Reporting
> >                                          (Host Reclaim in the case
> >                                           of virtualization)
> >
> > ---
> >
> > Patch Series Structure
> >
> > Patch 1/6: mm/hugetlb: add Kconfig and basic cache infrastructure
> >   - Introduces the CONFIG_HUGETLB_CACHE option, hstate tracking fields,
> >     and the HPG_cached page flag.
> >   - Establishes the clean helper API (hugetlb_folio_is_cached(),
> >     hugetlb_cache_remove(), hugetlb_cache_add()) and updates
> >     remove_hugetlb_folio() to be cache-aware, eliminating inline #ifdef
> >     blocks.
> > Patch 2/6: mm/hugetlb: implement cache recycling and allocation
> >   - Hooks up recycling in free_huge_folio() (up to the cache limit) and
> >     allocation in alloc_surplus_hugetlb_folio().
> >   - Implements MRU allocation policy for maximum warmth, poison safety
> >     checks, and MTE/dcache cleaning.
> > Patch 3/6: mm/hugetlb: add sysfs interfaces for cache
> >   - Exposes global and per-node sysfs attributes (max_cached_huge_pages,
> >     nr_cached_hugepages) for dynamic userspace control, including NUMA
> >     memory policy scaling.
> >   - Supports dynamic delta adjustments (+1/-1) to safely scale cache
> >     sizes alongside concurrent background reclaim operations.
> > Patch 4/6: mm/hugetlb: add memory shrinker for cache
> >   - Registers a NUMA-aware kernel shrinker to evict and dissolve cached
> >     gigantic pages back to buddy under memory pressure.
> > Patch 5/6: Documentation/admin-guide/mm/hugetlbpage.rst: document cache
> >     interfaces
> >   - Documents the Kconfig option, sysfs attributes, and shrinker
> >     behavior in the admin guide.
> > Patch 6/6: mm/hugetlb: support free page reporting for cached hugepages
> >   - Integrates the cache with the Free Page Reporting framework (virtio-
> >     balloon), introducing the HPG_reported flag and the
> >     reporting/isolation/draining lifecycle.
> > Sourav Panda (6):
> >   mm/hugetlb: add Kconfig and basic cache infrastructure
> >   mm/hugetlb: implement cache recycling and allocation
> >   mm/hugetlb: add sysfs interfaces for cache
> >   mm/hugetlb: add memory shrinker for cache
> >   Documentation/admin-guide/mm/hugetlbpage.rst: document cache
> >     interfaces
> >   mm/hugetlb: support free page reporting for cached hugepages
> >
> >  Documentation/admin-guide/mm/hugetlbpage.rst |  34 +-
> >  fs/Kconfig                                   |   9 +
> >  include/linux/hugetlb.h                      |  45 ++
> >  include/linux/page_reporting.h               |   1 +
> >  mm/hugetlb.c                                 | 590 ++++++++++++++++++-
> >  mm/hugetlb_internal.h                        |   9 +
> >  mm/hugetlb_sysfs.c                           | 158 +++++
> >  mm/page_reporting.c                          |  10 +-
> >  mm/page_reporting.h                          |   6 +
> >  9 files changed, 844 insertions(+), 18 deletions(-)
> >
> > --
> > 2.55.0.rc0.799.gd6f94ed593-goog
>

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[David Hildenbrand (Arm)]

On 7/7/26 08:42, Sourav Panda wrote:
> Overview
> This patch series introduces a dynamic, NUMA-aware HugePage Cache,
> backed by a kernel shrinker to safely return memory under pressure, and
> integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
> specifically targeting gigantic (1GB) hugepages. The goal is to solve
> the tradeoff between allocation latency and memory
> fungibility in virtualized and heterogeneous cloud environments.

Hi,

in general, we consider hugetlb nowadays to be mostly in feature freeze, as we
realized a while ago that adding more special casing on top of something too
special for all of MM is only going to hurt us more in the long run.

We want to have less special casing and less special sauce, not more.

Now, there is nothing wrong in making hugetlb be less special, by making it use
more of core infrastructure etc.

But optimizing for surplus hugetlb pages by teaching hugetlb about new caches
and its custom free-page-reporting support rather looks like the wrong direction
for me?

-- 
Cheers,

David

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Muchun Song]

> On Jul 7, 2026, at 15:29, David Hildenbrand (Arm) <david@kernel.org> wrote:
> 
> On 7/7/26 08:42, Sourav Panda wrote:
>> Overview
>> This patch series introduces a dynamic, NUMA-aware HugePage Cache,
>> backed by a kernel shrinker to safely return memory under pressure, and
>> integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
>> specifically targeting gigantic (1GB) hugepages. The goal is to solve
>> the tradeoff between allocation latency and memory
>> fungibility in virtualized and heterogeneous cloud environments.
> 
> Hi,
> 
> in general, we consider hugetlb nowadays to be mostly in feature freeze, as we
> realized a while ago that adding more special casing on top of something too
> special for all of MM is only going to hurt us more in the long run.
> 
> We want to have less special casing and less special sauce, not more.
> 
> Now, there is nothing wrong in making hugetlb be less special, by making it use
> more of core infrastructure etc.

+1

HugeTLB is already complex today, and the last thing we want is to compound
this complexity by piling more features on top of it. I fully share your view
that we should avoid going down this direction.

Thanks,
Muchun

> 
> But optimizing for surplus hugetlb pages by teaching hugetlb about new caches
> and its custom free-page-reporting support rather looks like the wrong direction
> for me?
> 
> -- 
> Cheers,
> 
> David

Re: [PATCH 6/6] mm/hugetlb: support free page reporting for cached hugepages

[Michael S. Tsirkin]

On Tue, Jul 07, 2026 at 06:42:35AM +0000, Sourav Panda wrote:
> Implement free page reporting for the HugeTLB dynamic cache.
> 
> Register HugeTLB hstates with the page reporting framework if they
> support caching.
> 
> When pages are added to the cache (either via recycling in
> free_huge_folio or direct population via sysfs nr_cached_hugepages),
> trigger a page reporting cycle.
> 
> Page reporting isolates pages from the cache list, reports them to the
> hypervisor via virtio-balloon, and then drains them back to the cache
> list, marking them as reported (HPG_reported).
> 
> If a page is allocated from the cache, or reclaimed, clear the reported
> flag.
> 
> Signed-off-by: Sourav Panda <souravpanda@google.com>

So if you are going to look into this, I feel the 1st step
should be reporting pages from hugepage_freelists - this
can be done independently.

-- 
MST

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Michael S. Tsirkin]

On Tue, Jul 07, 2026 at 09:29:07AM +0200, David Hildenbrand (Arm) wrote:
> On 7/7/26 08:42, Sourav Panda wrote:
> > Overview
> > This patch series introduces a dynamic, NUMA-aware HugePage Cache,
> > backed by a kernel shrinker to safely return memory under pressure, and
> > integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
> > specifically targeting gigantic (1GB) hugepages. The goal is to solve
> > the tradeoff between allocation latency and memory
> > fungibility in virtualized and heterogeneous cloud environments.
> 
> Hi,
> 
> in general, we consider hugetlb nowadays to be mostly in feature freeze, as we
> realized a while ago that adding more special casing on top of something too
> special for all of MM is only going to hurt us more in the long run.
> 
> We want to have less special casing and less special sauce, not more.
> 
> Now, there is nothing wrong in making hugetlb be less special, by making it use
> more of core infrastructure etc.
> 
> But optimizing for surplus hugetlb pages by teaching hugetlb about new caches
> and its custom free-page-reporting support rather looks like the wrong direction
> for me?
> 
> -- 
> Cheers,
> 
> David


It is currently bypassing free-page-reporting completely.
Making existing free lists not ignore free-page-reporting would
maybe considered "making it be less special"?

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[David Hildenbrand (Arm)]

On 7/7/26 12:25, Michael S. Tsirkin wrote:
> On Tue, Jul 07, 2026 at 09:29:07AM +0200, David Hildenbrand (Arm) wrote:
>> On 7/7/26 08:42, Sourav Panda wrote:
>>> Overview
>>> This patch series introduces a dynamic, NUMA-aware HugePage Cache,
>>> backed by a kernel shrinker to safely return memory under pressure, and
>>> integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
>>> specifically targeting gigantic (1GB) hugepages. The goal is to solve
>>> the tradeoff between allocation latency and memory
>>> fungibility in virtualized and heterogeneous cloud environments.
>>
>> Hi,
>>
>> in general, we consider hugetlb nowadays to be mostly in feature freeze, as we
>> realized a while ago that adding more special casing on top of something too
>> special for all of MM is only going to hurt us more in the long run.
>>
>> We want to have less special casing and less special sauce, not more.
>>
>> Now, there is nothing wrong in making hugetlb be less special, by making it use
>> more of core infrastructure etc.
>>
>> But optimizing for surplus hugetlb pages by teaching hugetlb about new caches
>> and its custom free-page-reporting support rather looks like the wrong direction
>> for me?
>>
>> -- 
>> Cheers,
>>
>> David
> 
> 
> It is currently bypassing free-page-reporting completely.
> Making existing free lists not ignore free-page-reporting would
> maybe considered "making it be less special"?
> 

Depends. We don't really want an orthogonal implementation of something we have
in core-mm.

-- 
Cheers,

David

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Lorenzo Stoakes]

On Tue, Jul 07, 2026 at 12:28:44PM +0200, David Hildenbrand (Arm) wrote:
> On 7/7/26 12:25, Michael S. Tsirkin wrote:
> > On Tue, Jul 07, 2026 at 09:29:07AM +0200, David Hildenbrand (Arm) wrote:
> >> On 7/7/26 08:42, Sourav Panda wrote:
> >>> Overview
> >>> This patch series introduces a dynamic, NUMA-aware HugePage Cache,
> >>> backed by a kernel shrinker to safely return memory under pressure, and
> >>> integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
> >>> specifically targeting gigantic (1GB) hugepages. The goal is to solve
> >>> the tradeoff between allocation latency and memory
> >>> fungibility in virtualized and heterogeneous cloud environments.
> >>
> >> Hi,
> >>
> >> in general, we consider hugetlb nowadays to be mostly in feature freeze, as we
> >> realized a while ago that adding more special casing on top of something too
> >> special for all of MM is only going to hurt us more in the long run.
> >>
> >> We want to have less special casing and less special sauce, not more.
> >>
> >> Now, there is nothing wrong in making hugetlb be less special, by making it use
> >> more of core infrastructure etc.
> >>
> >> But optimizing for surplus hugetlb pages by teaching hugetlb about new caches
> >> and its custom free-page-reporting support rather looks like the wrong direction
> >> for me?
> >>
> >> --
> >> Cheers,
> >>
> >> David
> >
> >
> > It is currently bypassing free-page-reporting completely.
> > Making existing free lists not ignore free-page-reporting would
> > maybe considered "making it be less special"?
> >

You have this completely backwards.

You're advocating making hugetlb _more special_ by duplicating functionality that
core mm already supports.

I mean:

mm/hugetlb.c                                 | 590 ++++++++++++++++++-

Tells the whole story right?

The whole issue with hugetlb is the very fact that it's a parallel
implementation of a bunch of mm stuff in its own little world.

We make it less special by mm/hugetlb.c smaller and smaller and implementing
what it does sanely elsewhere in _core mm_.

>
> Depends. We don't really want an orthogonal implementation of something we have
> in core-mm.

Yes, exactly.

Feature freeze means feature freeze, not 'feature that core mm doesn't support
feature freeze'.

Hugetlb is a poster child for poor decision making in mm that has left us
saddled with maintenance nightmares because we allowed 'just one more feature
in' (TM) with little to no thought to the future.

And we've all learned from that and don't want to repeat these kinds of
mistakes, nor make existing mistakes worse.

And work to improve hugetlbfs and make changes like the above are VERY welcome
:)

Laying a foundation for hugetlbfs to be more of a sane mm citizen through rework
series is really the asking price for stuff like this in my opinion.

>
> --
> Cheers,
>
> David

Thanks, Lorenzo

Re: [RFC PATCH 0/6] mm/hugetlb: Dynamic, NUMA-aware HugePage Cache & Free Page Reporting

[Michael S. Tsirkin]

On Tue, Jul 07, 2026 at 12:06:34PM +0100, Lorenzo Stoakes wrote:
> On Tue, Jul 07, 2026 at 12:28:44PM +0200, David Hildenbrand (Arm) wrote:
> > On 7/7/26 12:25, Michael S. Tsirkin wrote:
> > > On Tue, Jul 07, 2026 at 09:29:07AM +0200, David Hildenbrand (Arm) wrote:
> > >> On 7/7/26 08:42, Sourav Panda wrote:
> > >>> Overview
> > >>> This patch series introduces a dynamic, NUMA-aware HugePage Cache,
> > >>> backed by a kernel shrinker to safely return memory under pressure, and
> > >>> integrates it with Free Page Reporting (virtio-balloon) for HugeTLB,
> > >>> specifically targeting gigantic (1GB) hugepages. The goal is to solve
> > >>> the tradeoff between allocation latency and memory
> > >>> fungibility in virtualized and heterogeneous cloud environments.
> > >>
> > >> Hi,
> > >>
> > >> in general, we consider hugetlb nowadays to be mostly in feature freeze, as we
> > >> realized a while ago that adding more special casing on top of something too
> > >> special for all of MM is only going to hurt us more in the long run.
> > >>
> > >> We want to have less special casing and less special sauce, not more.
> > >>
> > >> Now, there is nothing wrong in making hugetlb be less special, by making it use
> > >> more of core infrastructure etc.
> > >>
> > >> But optimizing for surplus hugetlb pages by teaching hugetlb about new caches
> > >> and its custom free-page-reporting support rather looks like the wrong direction
> > >> for me?
> > >>
> > >> --
> > >> Cheers,
> > >>
> > >> David
> > >
> > >
> > > It is currently bypassing free-page-reporting completely.
> > > Making existing free lists not ignore free-page-reporting would
> > > maybe considered "making it be less special"?
> > >
> 
> You have this completely backwards.
> 
> You're advocating making hugetlb _more special_ by duplicating functionality that
> core mm already supports.
> 
> I mean:
> 
> mm/hugetlb.c                                 | 590 ++++++++++++++++++-
> 
> Tells the whole story right?
> 
> The whole issue with hugetlb is the very fact that it's a parallel
> implementation of a bunch of mm stuff in its own little world.
> 
> We make it less special by mm/hugetlb.c smaller and smaller and implementing
> what it does sanely elsewhere in _core mm_.
> 
> >
> > Depends. We don't really want an orthogonal implementation of something we have
> > in core-mm.
> 
> Yes, exactly.
> 
> Feature freeze means feature freeze, not 'feature that core mm doesn't support
> feature freeze'.
> 
> Hugetlb is a poster child for poor decision making in mm that has left us
> saddled with maintenance nightmares because we allowed 'just one more feature
> in' (TM) with little to no thought to the future.
> 
> And we've all learned from that and don't want to repeat these kinds of
> mistakes, nor make existing mistakes worse.
> 
> And work to improve hugetlbfs and make changes like the above are VERY welcome
> :)

Fair enough.

> Laying a foundation for hugetlbfs to be more of a sane mm citizen through rework
> series is really the asking price for stuff like this in my opinion.

Lorenzo, do you know *how* you want it reworked? Could you write it up
at a high level?
Because if not, it's not really practical to make it the asking price.


> >
> > --
> > Cheers,
> >
> > David
> 
> Thanks, Lorenzo