Re: [PATCH v3] mm/page_alloc: replace kernel_init_pages() with batch page clearing

["Salunke, Hrushikesh" <hsalunke@amd.com>]

On 24-04-2026 14:22, David Hildenbrand (Arm) wrote:
> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>
>
> On 4/24/26 10:42, Salunke, Hrushikesh wrote:
>> On 23-04-2026 16:42, Andrew Morton wrote:
>>> Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
>>>
>>>
>>> On Wed, 22 Apr 2026 10:26:58 +0000 Hrushikesh Salunke <hsalunke@amd.com> wrote:
>>>
>>>> When init_on_alloc is enabled, kernel_init_pages() clears every page
>>>> one at a time via clear_highpage_kasan_tagged(), which incurs per-page
>>>> kmap_local_page()/kunmap_local() overhead and prevents the architecture
>>>> clearing primitive from operating on contiguous ranges.
>>>>
>>>> Introduce clear_highpages_kasan_tagged() in highmem.h, a batch
>>>> clearing helper that calls clear_pages() for the full contiguous range
>>>> on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing
>>>> a single invocation of the arch clearing primitive across the entire
>>>> allocation. The HIGHMEM path falls back to per-page clearing since
>>>> those pages require kmap.
>>>>
>>>> Replace kernel_init_pages() with direct calls to the new helper, as it
>>>> becomes a trivial wrapper.
>>>>
>>>> Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1:
>>>>
>>>>   Before: 0.445s
>>>>   After:  0.166s  (-62.7%, 2.68x faster)
>>> Nice.
>>>
>>>> Kernel time (sys) reduction per workload with init_on_alloc=1:
>>>>
>>>>   Workload            Before       After       Change
>>>>   Graph500 64C128T    30m 41.8s    15m 14.8s   -50.3%
>>>>   Graph500 16C32T     15m 56.7s     9m 43.7s   -39.0%
>>>>   Pagerank 32T         1m 58.5s     1m 12.8s   -38.5%
>>>>   Pagerank 128T        2m 36.3s     1m 40.4s   -35.7%
>>>>
>>>> ...
>>>>
>>>> --- a/include/linux/highmem.h
>>>> +++ b/include/linux/highmem.h
>>>> @@ -345,6 +345,21 @@ static inline void clear_highpage_kasan_tagged(struct page *page)
>>>>       kunmap_local(kaddr);
>>>>  }
>>>>
>>>> +static inline void clear_highpages_kasan_tagged(struct page *page, int numpages)
>>>> +{
>>>> +     /* s390's use of memset() could override KASAN redzones. */
>>>> +     kasan_disable_current();
>>>> +     if (!IS_ENABLED(CONFIG_HIGHMEM)) {
>>>> +             clear_pages(kasan_reset_tag(page_address(page)), numpages);
>>>> +     } else {
>>>> +             int i;
>>>> +
>>>> +             for (i = 0; i < numpages; i++)
>>>> +                     clear_highpage_kasan_tagged(page + i);
>>>> +     }
>>>> +     kasan_enable_current();
>>>> +}
>>> Why was it globally published and inlined?  Is there any expectation
>>> that this will be used outside of page_alloc.c?
>>>
>>> Both of the callsites are themselves inlined.  The patch adds 330 bytes
>>> to my arm allmodcnfig page_alloc.o - did we gain anything from that?
>>>
>> Hi Andrew,
>>
>> The idea was to keep it alongside clear_highpage_kasan_tagged() as its
>> batch counterpart, but currently it is only used by page_alloc.c.
> Right.
>
> Looking at init_vmalloc_pages(), I wonder if it could also benefit from batching
> if we find that pages are actually contiguous.
>
> That would require looking up multiple pages at once. vmalloc_to_pages() or sth
> like that. Surely, doing such an optimized page table walk could be beneficial
> by itself.

Interesting idea. For the general case where we only have struct page
pointers, we'd need physical contiguity detection and a batched page
table walk as you described. But looking at init_vmalloc_pages() 
specifically, it already has the vmalloc virtual address which is 
contiguous, so can we just do following and potentially skip the 
vmalloc_to_page() walk entirely:

clear_pages(kasan_reset_tag((void *)start), size >> PAGE_SHIFT);

What do you think? would this simpler approach work
, or am I missing something?

>
>> Your concern about the code size increase is valid. Would you prefer if
>> I move it to page_alloc.c as a static function and drop the inline
>> in v4? If an external user comes along later it can always be moved
>> back to the header.
> What is exactly is responsible for the code increase? Two calls in
> clear_highpages_kasan_tagged()?
>
> Surely the compiler would just inline kernel_init_pages() already?
>
> So my best guess that the 330 bytes are just clear_pages() overhead or some code
> layout changes?

You're right, it's essentially the clear_pages() overhead being
duplicated at each call site. The compiler was actually not inlining
kernel_init_pages(), it was a standalone function. But it was inlining
post_alloc_hook() and free_pages_prepare() into their callers, so with
the patch each of those inlined copies now carries the full 
clear_highpages_kasan_tagged() code instead of a small call instruction.

I ran bloat-o-meter on arm allmodconfig and confirmed this. I also
tested moving clear_highpages_kasan_tagged() into page_alloc.c as 
a static (non-inline) function, and the bloat disappears entirely. As
currently there are no other users of this function so I will move it
in page_alloc.c. I will make this change in v4.

Regards,
Hrushikesh.