[PATCH 00/49] mm: Generalize vmemmap optimization for DAX and HugeTLB

Muchun Song
2026-04-05T12:53:27Z

Overview:
This patch series generalizes the HugeTLB Vmemmap Optimization (HVO)
into a generic vmemmap optimization framework that can be used by both
HugeTLB and DAX.

Background:
Currently, the vmemmap optimization feature is highly coupled with
HugeTLB. However, DAX also has similar requirements for optimizing vmemmap
pages to save memory. The current implementation has separate vmemmap
optimization paths for HugeTLB and DAX, leading to duplicated logic,
complex initialization sequences, and architecture-specific flags.

Implementation:
This series breaks down the optimization into a generic framework:
- Patch 1-6: Fix bugs related to sparse vmemmap initialization and DAX.
- Patch 7-13: Refactor the existing sparse vmemmap initialization.
- Patch 14-26: Decouple the vmemmap optimization from HugeTLB and
introduce generic optimization macros and functions.
- Patch 27-39: Switch HugeTLB and DAX to use the generic framework.
- Patch 40-49: Clean up the old HVO-specific code and simplify it.

Benifit:
- When CONFIG_DEFERRED_STRUCT_PAGE_INIT is disabled, all struct pages
utilizing HVO (HugeTLB Vmemmap Optimization) skip initialization in
memmap_init, significantly accelerating boot times.
- All architectures supporting HVO benefit from the optimizations
provided by SPARSEMEM_VMEMMAP_PREINIT without requiring
architecture-specific adaptations.
- Device DAX struct page savings are further improved, saving an
additional 4KB of struct page memory for every 2MB huge page.
- Vmemmap tail pages used for Device DAX shared mappings are changed
from read-write to read-only, enhancing system security.
- HugeTLB and Device DAX now share a unified vmemmap optimization
framework, reducing long-term maintenance overhead.

Testing:
- Verification: Compiled and tested on x86 architecture.

Chengkaitao (1):
mm: Convert vmemmap_p?d_populate() to static functions

Muchun Song (48):
mm/sparse: fix vmemmap accounting imbalance on memory hotplug error
mm/sparse: add a @pgmap argument to memory deactivation paths
mm/sparse: fix vmemmap page accounting for HVOed DAX
mm/sparse: add a @pgmap parameter to arch vmemmap_populate()
mm/sparse: fix missing architecture-specific page table sync for HVO
DAX
mm/mm_init: fix uninitialized pageblock migratetype for ZONE_DEVICE
compound pages
mm/mm_init: use pageblock_migratetype_init_range() in
deferred_free_pages()
mm: panic on memory allocation failure in sparse_init_nid()
mm: move subsection_map_init() into sparse_init()
mm: defer sparse_init() until after zone initialization
mm: make set_pageblock_order() static
mm: integrate sparse_vmemmap_init_nid_late() into sparse_init_nid()
mm/cma: validate hugetlb CMA range by zone at reserve time
mm/hugetlb: free cross-zone bootmem gigantic pages after allocation
mm/hugetlb: initialize vmemmap optimization in early stage
mm: remove sparse_vmemmap_init_nid_late()
mm/mm_init: make __init_page_from_nid() static
mm/sparse-vmemmap: remove the VMEMMAP_POPULATE_PAGEREF flag
mm: rename vmemmap optimization macros to generic names
mm/sparse: drop power-of-2 size requirement for struct mem_section
mm/sparse: introduce compound page order to mem_section
mm/mm_init: skip initializing shared tail pages for compound pages
mm/sparse-vmemmap: initialize shared tail vmemmap page upon allocation
mm/sparse-vmemmap: support vmemmap-optimizable compound page
population
mm/hugetlb: use generic vmemmap optimization macros
mm: call memblocks_present() before HugeTLB initialization
mm/hugetlb: switch HugeTLB to use generic vmemmap optimization
mm: extract pfn_to_zone() helper
mm/sparse-vmemmap: remove unused SPARSEMEM_VMEMMAP_PREINIT feature
mm/hugetlb: remove HUGE_BOOTMEM_HVO flag and simplify pre-HVO logic
mm/sparse-vmemmap: consolidate shared tail page allocation
mm: introduce CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION
mm/sparse-vmemmap: switch DAX to use generic vmemmap optimization
mm/sparse-vmemmap: introduce section zone to struct mem_section
powerpc/mm: use generic vmemmap_shared_tail_page() in compound vmemmap
mm/sparse-vmemmap: unify DAX and HugeTLB vmemmap optimization
mm/sparse-vmemmap: remap the shared tail pages as read-only
mm/sparse-vmemmap: remove unused ptpfn argument
mm/hugetlb_vmemmap: remove vmemmap_wrprotect_hvo() and related code
mm/sparse: simplify section_vmemmap_pages()
mm/sparse-vmemmap: introduce section_vmemmap_page_structs()
powerpc/mm: rely on generic vmemmap_can_optimize() to simplify code
mm/sparse-vmemmap: drop ARCH_WANT_OPTIMIZE_DAX_VMEMMAP and simplify
checks
mm/sparse-vmemmap: drop @pgmap parameter from vmemmap populate APIs
mm/sparse: replace pgmap with order and zone in sparse_add_section()
mm: redefine HVO as Hugepage Vmemmap Optimization
Documentation/mm: restructure vmemmap_dedup.rst to reflect generalized
HVO
mm: consolidate struct page power-of-2 size checks for HVO

.../admin-guide/kernel-parameters.txt | 2 +-
Documentation/admin-guide/sysctl/vm.rst | 2 +-
Documentation/mm/vmemmap_dedup.rst | 218 ++--------
arch/powerpc/Kconfig | 1 -
arch/powerpc/include/asm/book3s/64/radix.h | 12 -
arch/powerpc/mm/book3s64/radix_pgtable.c | 114 +----
arch/powerpc/mm/init_64.c | 1 +
arch/riscv/Kconfig | 1 -
arch/x86/Kconfig | 2 -
fs/Kconfig | 6 +-
include/linux/hugetlb.h | 7 +-
include/linux/memory_hotplug.h | 2 +-
include/linux/mm.h | 50 +--
include/linux/mm_types.h | 2 +
include/linux/mm_types_task.h | 4 +
include/linux/mmzone.h | 143 ++++---
include/linux/page-flags.h | 31 +-
kernel/bounds.c | 2 +
mm/Kconfig | 20 +-
mm/bootmem_info.c | 5 +-
mm/cma.c | 3 +-
mm/hugetlb.c | 143 +++----
mm/hugetlb_cma.c | 3 +-
mm/hugetlb_vmemmap.c | 237 +----------
mm/hugetlb_vmemmap.h | 35 +-
mm/internal.h | 26 +-
mm/memory_hotplug.c | 15 +-
mm/mm_init.c | 138 +++---
mm/sparse-vmemmap.c | 392 +++++-------------
mm/sparse.c | 85 ++--
mm/util.c | 2 +-
scripts/gdb/linux/mm.py | 6 +-
32 files changed, 513 insertions(+), 1197 deletions(-)

--
2.20.1