[PATCH] mm/page_alloc: use batch page clearing in kernel_init_pages()

Hrushikesh Salunke
2026-04-08T09:25:24Z

When init_on_alloc is enabled, kernel_init_pages() clears every page
one at a time, calling clear_page() per page. This is unnecessarily
slow for large contiguous allocations (mTHPs, HugeTLB) that dominate
real workloads.

On 64-bit (!HIGHMEM) systems, switch to clearing pages in batch via
clear_pages(), bypassing the per-page kmap_local_page()/kunmap_local()
overhead and allowing the arch clearing primitive to operate on the full
contiguous range in a single invocation. The batch size is the full
allocation when the preempt model is preemptible (preemption points are
implicit), or PROCESS_PAGES_NON_PREEMPT_BATCH otherwise, with
cond_resched() between batches to limit scheduling latency under
cooperative preemption.

The HIGHMEM path is kept as-is since those pages require kmap.

Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1:

Before: 0.445s
After: 0.166s (-62.7%, 2.68x faster)

Kernel time (sys) reduction per workload with init_on_alloc=1:

Workload Before After Change
Graph500 64C128T 30m 41.8s 15m 14.8s -50.3%
Graph500 16C32T 15m 56.7s 9m 43.7s -39.0%
Pagerank 32T 1m 58.5s 1m 12.8s -38.5%
Pagerank 128T 2m 36.3s 1m 40.4s -35.7%

Signed-off-by: Hrushikesh Salunke <hsalunke@amd.com>
---
base commit: 1a2fbbe3653f0ebb24af9b306a8a968287344a35

mm/page_alloc.c | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index b1c5430cad4e..178cbebadd50 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1224,8 +1224,23 @@ static void kernel_init_pages(struct page *page, int numpages)

/* s390's use of memset() could override KASAN redzones. */
kasan_disable_current();
- for (i = 0; i < numpages; i++)
- clear_highpage_kasan_tagged(page + i);
+
+ if (!IS_ENABLED(CONFIG_HIGHMEM)) {
+ void *addr = kasan_reset_tag(page_address(page));
+ unsigned int unit = preempt_model_preemptible() ?
+ numpages : PROCESS_PAGES_NON_PREEMPT_BATCH;
+ int count;
+
+ for (i = 0; i < numpages; i += count) {
+ cond_resched();
+ count = min_t(int, unit, numpages - i);
+ clear_pages(addr + (i << PAGE_SHIFT), count);
+ }
+ } else {
+ for (i = 0; i < numpages; i++)
+ clear_highpage_kasan_tagged(page + i);
+ }
+
kasan_enable_current();
}

--
2.43.0