Re: [PATCH v3] mm/page_alloc: replace kernel_init_pages() with batch page clearing

["David Hildenbrand (Arm)" <david@kernel.org>]

On 4/24/26 10:42, Salunke, Hrushikesh wrote:
> 
> On 23-04-2026 16:42, Andrew Morton wrote:
>> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>>
>>
>> On Wed, 22 Apr 2026 10:26:58 +0000 Hrushikesh Salunke <hsalunke@amd.com> wrote:
>>
>>> When init_on_alloc is enabled, kernel_init_pages() clears every page
>>> one at a time via clear_highpage_kasan_tagged(), which incurs per-page
>>> kmap_local_page()/kunmap_local() overhead and prevents the architecture
>>> clearing primitive from operating on contiguous ranges.
>>>
>>> Introduce clear_highpages_kasan_tagged() in highmem.h, a batch
>>> clearing helper that calls clear_pages() for the full contiguous range
>>> on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing
>>> a single invocation of the arch clearing primitive across the entire
>>> allocation. The HIGHMEM path falls back to per-page clearing since
>>> those pages require kmap.
>>>
>>> Replace kernel_init_pages() with direct calls to the new helper, as it
>>> becomes a trivial wrapper.
>>>
>>> Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1:
>>>
>>>   Before: 0.445s
>>>   After:  0.166s  (-62.7%, 2.68x faster)
>> Nice.
>>
>>> Kernel time (sys) reduction per workload with init_on_alloc=1:
>>>
>>>   Workload            Before       After       Change
>>>   Graph500 64C128T    30m 41.8s    15m 14.8s   -50.3%
>>>   Graph500 16C32T     15m 56.7s     9m 43.7s   -39.0%
>>>   Pagerank 32T         1m 58.5s     1m 12.8s   -38.5%
>>>   Pagerank 128T        2m 36.3s     1m 40.4s   -35.7%
>>>
>>> ...
>>>
>>> --- a/include/linux/highmem.h
>>> +++ b/include/linux/highmem.h
>>> @@ -345,6 +345,21 @@ static inline void clear_highpage_kasan_tagged(struct page *page)
>>>       kunmap_local(kaddr);
>>>  }
>>>
>>> +static inline void clear_highpages_kasan_tagged(struct page *page, int numpages)
>>> +{
>>> +     /* s390's use of memset() could override KASAN redzones. */
>>> +     kasan_disable_current();
>>> +     if (!IS_ENABLED(CONFIG_HIGHMEM)) {
>>> +             clear_pages(kasan_reset_tag(page_address(page)), numpages);
>>> +     } else {
>>> +             int i;
>>> +
>>> +             for (i = 0; i < numpages; i++)
>>> +                     clear_highpage_kasan_tagged(page + i);
>>> +     }
>>> +     kasan_enable_current();
>>> +}
>> Why was it globally published and inlined?  Is there any expectation
>> that this will be used outside of page_alloc.c?
>>
>> Both of the callsites are themselves inlined.  The patch adds 330 bytes
>> to my arm allmodcnfig page_alloc.o - did we gain anything from that?
>>
> Hi Andrew,
> 
> The idea was to keep it alongside clear_highpage_kasan_tagged() as its
> batch counterpart, but currently it is only used by page_alloc.c.

Right.

Looking at init_vmalloc_pages(), I wonder if it could also benefit from batching
if we find that pages are actually contiguous.

That would require looking up multiple pages at once. vmalloc_to_pages() or sth
like that. Surely, doing such an optimized page table walk could be beneficial
by itself.

> 
> Your concern about the code size increase is valid. Would you prefer if
> I move it to page_alloc.c as a static function and drop the inline
> in v4? If an external user comes along later it can always be moved
> back to the header.

What is exactly is responsible for the code increase? Two calls in
clear_highpages_kasan_tagged()?

Surely the compiler would just inline kernel_init_pages() already?

So my best guess that the 330 bytes are just clear_pages() overhead or some code
layout changes?

-- 
Cheers,

David