Re: [PATCH 00/11] mm/hugetlb: Eliminate fake head pages from vmemmap optimization

Frank van der Linden
2025-12-09T18:20:29Z

On Fri, Dec 5, 2025 at 11:44 AM Kiryl Shutsemau <kas@kernel.org> wrote:
>
> This series removes "fake head pages" from the HugeTLB vmemmap
> optimization (HVO) by changing how tail pages encode their relationship
> to the head page.
>
> It simplifies compound_head() and page_ref_add_unless(). Both are in the
> hot path.
>
> Background
> ==========
>
> HVO reduces memory overhead by freeing vmemmap pages for HugeTLB pages
> and remapping the freed virtual addresses to a single physical page.
> Previously, all tail page vmemmap entries were remapped to the first
> vmemmap page (containing the head struct page), creating "fake heads" -
> tail pages that appear to have PG_head set when accessed through the
> deduplicated vmemmap.
>
> This required special handling in compound_head() to detect and work
> around fake heads, adding complexity and overhead to a very hot path.
>
> New Approach
> ============
>
> For architectures/configs where sizeof(struct page) is a power of 2 (the
> common case), this series changes how position of the head page is encoded
> in the tail pages.
>
> Instead of storing a pointer to the head page, the ->compound_info
> (renamed from ->compound_head) now stores a mask.
>
> The mask can be applied to any tail page's virtual address to compute
> the head page address. Critically, all tail pages of the same order now
> have identical compound_info values, regardless of which compound page
> they belong to.
>
> This enables a key optimization: instead of remapping tail vmemmap
> entries to the head page (creating fake heads), we remap them to a
> shared, pre-initialized vmemmap_tail page per hstate. The head page
> gets its own dedicated vmemmap page, eliminating fake heads entirely.
>
> Benefits
> ========
>
> 1. Smaller generated code. On defconfig, I see ~15K reduction of text
> in vmlinux:
>
> add/remove: 6/33 grow/shrink: 54/262 up/down: 6130/-21922 (-15792)
>
> 2. Simplified compound_head(): No fake head detection needed. The
> function is now branchless for power-of-2 struct page sizes.
>
> 3. Eliminated race condition: The old scheme required synchronize_rcu()
> to coordinate between HVO remapping and speculative PFN walkers that
> might write to fake heads. With the head page always in writable
> memory, this synchronization is unnecessary.
>
> 4. Removed static key: hugetlb_optimize_vmemmap_key is no longer needed
> since compound_head() no longer has HVO-specific branches.
>
> 5. Cleaner architecture: The vmemmap layout is now straightforward -
> head page has its own vmemmap, tails share a read-only template.
>
> I had hoped to see performance improvement, but my testing thus far has
> shown either no change or only a slight improvement within the noise.
>
> Series Organization
> ===================
>
> Patches 1-3: Preparatory refactoring
> - Change prep_compound_tail() interface to take order
> - Rename compound_head field to compound_info
> - Move set/clear_compound_head() near compound_head()
>
> Patch 4: Core encoding change
> - Implement mask-based encoding for power-of-2 struct page
>
> Patches 5-6: HVO restructuring
> - Refactor vmemmap_walk to support separate head/tail pages
> - Introduce per-hstate vmemmap_tail, eliminate fake heads
>
> Patches 7-9: Cleanup
> - Remove fake head checks from compound_head(), PageTail(), etc.
> - Remove VMEMMAP_SYNCHRONIZE_RCU and synchronize_rcu() calls
> - Remove hugetlb_optimize_vmemmap_key static key
>
> Patch 10: Optimization
> - Implement branchless compound_head() for power-of-2 case
>
> Patch 11: Documentation
> - Update vmemmap_dedup.rst to reflect new architecture
>
> Kiryl Shutsemau (11):
> mm: Change the interface of prep_compound_tail()
> mm: Rename the 'compound_head' field in the 'struct page' to
> 'compound_info'
> mm: Move set/clear_compound_head() to compound_head()
> mm: Rework compound_head() for power-of-2 sizeof(struct page)
> mm/hugetlb: Refactor code around vmemmap_walk
> mm/hugetlb: Remove fake head pages
> mm: Drop fake head checks and fix a race condition
> hugetlb: Remove VMEMMAP_SYNCHRONIZE_RCU
> mm/hugetlb: Remove hugetlb_optimize_vmemmap_key static key
> mm: Remove the branch from compound_head()
> hugetlb: Update vmemmap_dedup.rst
>
> .../admin-guide/kdump/vmcoreinfo.rst | 2 +-
> Documentation/mm/vmemmap_dedup.rst | 62 ++---
> include/linux/hugetlb.h | 3 +
> include/linux/mm_types.h | 20 +-
> include/linux/page-flags.h | 163 +++++-------
> include/linux/page_ref.h | 8 +-
> include/linux/types.h | 2 +-
> kernel/vmcore_info.c | 2 +-
> mm/hugetlb.c | 8 +-
> mm/hugetlb_vmemmap.c | 245 ++++++++----------
> mm/hugetlb_vmemmap.h | 4 +-
> mm/internal.h | 11 +-
> mm/mm_init.c | 2 +-
> mm/page_alloc.c | 4 +-
> mm/slab.h | 2 +-
> mm/util.c | 15 +-
> 16 files changed, 242 insertions(+), 311 deletions(-)
>
> --
> 2.51.2
>
>

I love this in general - I've always disliked the fake head
construction (though I understand the reason behind it).

However, it seems like you didn't add support to vmemmap_populate_hvo,
as far as I can tell. That's the function that is used to do HVO early
on bootmem (memblock) allocated 'gigantic' pages. So I think that
would break with this patch.

Could you add support there too? I don't think it would be hard to.
While at it, you could also do it for vmemmap_populate_hugepages to
support devdax :-)

- Frank