Re: [PATCH 08/17] mm/sparse: mark memory sections present earlier

Re: [PATCH 08/17] mm/sparse: mark memory sections present earlier

[Mike Rapoport]

On Thu, Jul 02, 2026 at 05:38:12PM +0800, Muchun Song wrote:
> Upcoming HugeTLB bootmem changes need sparsemem section metadata before
> the HugeTLB bootmem allocation path runs. The memblock ranges are marked
> present from sparse_init(), which is called too late for that setup.

It's not only that memblock regions are marked present, but it actually
initializes mem_section's for the present memory ...
 
> Move the code that marks memblock ranges present into
> mm_core_init_early(), before free_area_init() and the HugeTLB bootmem
> setup. Rename the helper to sparse_memblock_present() to make the new

... so let's name this function to reflect that.

How about sparse_sections_init()?

> caller describe the sparsemem-specific initialization step.
> 
> This is a preparatory change.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> ---
>  mm/internal.h | 2 ++
>  mm/mm_init.c  | 1 +
>  mm/sparse.c   | 4 +---
>  3 files changed, 4 insertions(+), 3 deletions(-)

-- 
Sincerely yours,
Mike.

Re: [PATCH 08/17] mm/sparse: mark memory sections present earlier

[Muchun Song]

> On Jul 9, 2026, at 18:54, Mike Rapoport <rppt@kernel.org> wrote:
> 
> On Thu, Jul 02, 2026 at 05:38:12PM +0800, Muchun Song wrote:
>> Upcoming HugeTLB bootmem changes need sparsemem section metadata before
>> the HugeTLB bootmem allocation path runs. The memblock ranges are marked
>> present from sparse_init(), which is called too late for that setup.
> 
> It's not only that memblock regions are marked present, but it actually
> initializes mem_section's for the present memory ...

Right.

> 
>> Move the code that marks memblock ranges present into
>> mm_core_init_early(), before free_area_init() and the HugeTLB bootmem
>> setup. Rename the helper to sparse_memblock_present() to make the new
> 
> ... so let's name this function to reflect that.
> 
> How about sparse_sections_init()?

Make sense. This name really captures the function's purpose well.
I'll go ahead and use it.

Muchun,
Thanks.

> 
>> caller describe the sparsemem-specific initialization step.
>> 
>> This is a preparatory change.
>> 
>> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>> ---
>> mm/internal.h | 2 ++
>> mm/mm_init.c  | 1 +
>> mm/sparse.c   | 4 +---
>> 3 files changed, 4 insertions(+), 3 deletions(-)
> 
> -- 
> Sincerely yours,
> Mike.

Re: [PATCH 02/17] mm/sparse-vmemmap: track compound page order in struct mem_section

[Mike Rapoport]

> HugeTLB and DAX both rely on vmemmap optimization, but sparsemem does
> not record what compound page order a section is populated with.
> 
> As a result, code that needs this information has to open-code
> separate handling across users of vmemmap optimization.  It also
> prevents other memory management code, such as struct page
> initialization, from skipping initialization of shared vmemmap pages
> when needed.
> 
> Track the compound page order in struct mem_section and provide small
> helpers to access it.  A compound page larger than a section naturally
> carries the same order across all covered sections.
> 
> This is a preparatory change for consolidating vmemmap optimization
> handling and for letting later code make initialization decisions
> based on the section's compound page order.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 1353bcf7b712..bacd89572c5c 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -2015,6 +2015,14 @@ struct mem_section {
>  	 */
>  	struct page_ext *page_ext;
>  #endif
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
> +	/*
> +	 * The order of compound pages in this section. Typically, the section
> +	 * holds compound pages of this order; a larger compound page will span
> +	 * multiple sections.
> +	 */

Typically a section holds order-0 pages ;-)

I think this comment should mention that this is for special sections,
like hugetlb and DAX.

> +	unsigned int order;
> +#endif
>  };
>  
>  #ifdef CONFIG_SPARSEMEM_EXTREME
> @@ -2361,10 +2369,34 @@ static inline unsigned long next_present_section_nr(unsigned long section_nr)
>  #endif
>  
>  #else
> +struct mem_section;
> +
>  #define sparse_vmemmap_init_nid_early(_nid) do {} while (0)
>  #define pfn_in_present_section pfn_valid
>  #endif /* CONFIG_SPARSEMEM */
>  
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
> +static inline void section_set_order(struct mem_section *section, unsigned int order)
> +{
> +	VM_WARN_ON(section->order && order && section->order != order);
> +	section->order = order;
> +}
> +
> +static inline unsigned int section_order(const struct mem_section *section)
> +{
> +	return section->order;
> +}

You mention open-coded handling, is there any open-coded case that can
be replaced with the new helpers right away?

We strive to have new APIs and at least some of their users in the same
patch.

-- 
Sincerely yours,
Mike.

Re: [PATCH 04/17] mm/mm_init: skip initializing shared vmemmap tail pages

[Mike Rapoport]

Hi Muchun,

Below are some preliminary comments, I'm planning to spend more time on
review next week.

On Thu, Jul 02, 2026 at 05:38:08PM +0800, Muchun Song wrote:
> memmap_init_range() initializes every struct page in the target range.
> For compound pages with vmemmap optimization, the tail struct pages are
> backed by a shared vmemmap page.
> 
> Initializing those tail struct pages would overwrite the shared
> vmemmap page contents, so users such as HugeTLB have to open-code
> follow-up handling to restore the metadata afterwards.
> 
> Use the section's compound page order to detect struct pages that fall
> into the shared tail vmemmap range and skip their initialization in
> memmap_init_range().  Still initialize the pageblock migratetypes for
> the skipped range so the surrounding setup remains intact.
> 
> This is a preparatory change for consolidating handling across users of
> vmemmap optimization, and it also avoids redundant initialization of
> shared tail vmemmap pages during early boot.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> ---
>  include/linux/mmzone.h |  4 ++++
>  mm/internal.h          | 16 ++++++++++++++++
>  mm/mm_init.c           | 25 +++++++++++++++++++------
>  3 files changed, 39 insertions(+), 6 deletions(-)
> 
> @@ -673,19 +673,21 @@ static inline void fixup_hashdist(void)
>  static inline void fixup_hashdist(void) {}
>  #endif /* CONFIG_NUMA */
>  
> -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
>  static __meminit void pageblock_migratetype_init_range(unsigned long pfn,
> -		unsigned long nr_pages, int migratetype, bool atomic)
> +		unsigned long nr_pages, int migratetype, bool isolate, bool atomic)

What is isolate parameter for?

>  {
>  	const unsigned long end = pfn + nr_pages;
>  
>  	for (pfn = pageblock_align(pfn); pfn < end; pfn += pageblock_nr_pages) {
> -		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false);
> +		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, isolate);
> +#ifdef CONFIG_SPARSEMEM
>  		if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION))
> +#else
> +		if (!atomic && IS_ALIGNED(pfn, MAX_FOLIO_NR_PAGES))
> +#endif

Let's trigger cond_resched() on some defined number of iterations or some
memory size chunk, e.g PAGES_PER_128M or even PAGES_PER_1G.

>  			cond_resched();
>  	}
>  }

-- 
Sincerely yours,
Mike.

Re: [PATCH 04/17] mm/mm_init: skip initializing shared vmemmap tail pages

[Muchun Song]

> On Jul 9, 2026, at 18:45, Mike Rapoport <rppt@kernel.org> wrote:
> 
> Hi Muchun,

Hi,

> 
> Below are some preliminary comments, I'm planning to spend more time on
> review next week.

Thanks for the early feedback! Looking forward to more review next week.

> 
> On Thu, Jul 02, 2026 at 05:38:08PM +0800, Muchun Song wrote:
>> memmap_init_range() initializes every struct page in the target range.
>> For compound pages with vmemmap optimization, the tail struct pages are
>> backed by a shared vmemmap page.
>> 
>> Initializing those tail struct pages would overwrite the shared
>> vmemmap page contents, so users such as HugeTLB have to open-code
>> follow-up handling to restore the metadata afterwards.
>> 
>> Use the section's compound page order to detect struct pages that fall
>> into the shared tail vmemmap range and skip their initialization in
>> memmap_init_range().  Still initialize the pageblock migratetypes for
>> the skipped range so the surrounding setup remains intact.
>> 
>> This is a preparatory change for consolidating handling across users of
>> vmemmap optimization, and it also avoids redundant initialization of
>> shared tail vmemmap pages during early boot.
>> 
>> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>> ---
>> include/linux/mmzone.h |  4 ++++
>> mm/internal.h          | 16 ++++++++++++++++
>> mm/mm_init.c           | 25 +++++++++++++++++++------
>> 3 files changed, 39 insertions(+), 6 deletions(-)
>> 
>> @@ -673,19 +673,21 @@ static inline void fixup_hashdist(void)
>> static inline void fixup_hashdist(void) {}
>> #endif /* CONFIG_NUMA */
>> 
>> -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
>> static __meminit void pageblock_migratetype_init_range(unsigned long pfn,
>> - unsigned long nr_pages, int migratetype, bool atomic)
>> + unsigned long nr_pages, int migratetype, bool isolate, bool atomic)
> 
> What is isolate parameter for?

I've re-examined the code, and you're right that the isolate parameter is technically
redundant for our current use case, as memmap_init_zone_range() passes false.

The rationale behind keeping it is future-proofing. The ultimate goal of a generic
HVO is to support arbitrary huge pages, not just HugeTLB. I decoupled this as a
parameter to prevent potential regressions down the road; if a developer leverages
this for other huge page types in the future, they won't inadvertently break
things by forgetting to update a hardcoded false in init_pageblock_migratetype(),
especially since memmap_init_range() natively accepts an isolate parameter.

Of course, we could also just delete this parameter for now and add it back later if
needed. I think both approaches work.

Which way are you leaning?

> 
>> {
>> 	const unsigned long end = pfn + nr_pages;
>> 
>> 	for (pfn = pageblock_align(pfn); pfn < end; pfn += pageblock_nr_pages) {
>> - 		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false);
>> + 		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, isolate);
>> +#ifdef CONFIG_SPARSEMEM
>> 		if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION))
>> +#else
>> + 		if (!atomic && IS_ALIGNED(pfn, MAX_FOLIO_NR_PAGES))
>> +#endif
> 
> Let's trigger cond_resched() on some defined number of iterations or some
> memory size chunk, e.g PAGES_PER_128M or even PAGES_PER_1G.

Yes, that's for the best. I was really struggling to choose a suitable macro
for this earlier, but I realized it's a difficult thing to get right. I'm leaning
toward selecting PAGES_PER_1G instead.

Muchun,
Thanks.

> 
>> 			cond_resched();
>> 	}
>> }
> 
> -- 
> Sincerely yours,
> Mike.

Re: [PATCH 04/17] mm/mm_init: skip initializing shared vmemmap tail pages

[Mike Rapoport]

On Thu, Jul 09, 2026 at 08:31:31PM +0800, Muchun Song wrote:
> > On Jul 9, 2026, at 18:45, Mike Rapoport <rppt@kernel.org> wrote:
> > On Thu, Jul 02, 2026 at 05:38:08PM +0800, Muchun Song wrote:
> >> memmap_init_range() initializes every struct page in the target range.
> >> For compound pages with vmemmap optimization, the tail struct pages are
> >> backed by a shared vmemmap page.
> >> 
> >> Initializing those tail struct pages would overwrite the shared
> >> vmemmap page contents, so users such as HugeTLB have to open-code
> >> follow-up handling to restore the metadata afterwards.
> >> 
> >> Use the section's compound page order to detect struct pages that fall
> >> into the shared tail vmemmap range and skip their initialization in
> >> memmap_init_range().  Still initialize the pageblock migratetypes for
> >> the skipped range so the surrounding setup remains intact.
> >> 
> >> This is a preparatory change for consolidating handling across users of
> >> vmemmap optimization, and it also avoids redundant initialization of
> >> shared tail vmemmap pages during early boot.
> >> 
> >> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
> >> ---
> >> include/linux/mmzone.h |  4 ++++
> >> mm/internal.h          | 16 ++++++++++++++++
> >> mm/mm_init.c           | 25 +++++++++++++++++++------
> >> 3 files changed, 39 insertions(+), 6 deletions(-)
> >> 
> >> @@ -673,19 +673,21 @@ static inline void fixup_hashdist(void)
> >> static inline void fixup_hashdist(void) {}
> >> #endif /* CONFIG_NUMA */
> >> 
> >> -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
> >> static __meminit void pageblock_migratetype_init_range(unsigned long pfn,
> >> - unsigned long nr_pages, int migratetype, bool atomic)
> >> + unsigned long nr_pages, int migratetype, bool isolate, bool atomic)
> > 
> > What is isolate parameter for?
> 
> I've re-examined the code, and you're right that the isolate parameter is technically
> redundant for our current use case, as memmap_init_zone_range() passes false.
> 
> The rationale behind keeping it is future-proofing. The ultimate goal of a generic
> HVO is to support arbitrary huge pages, not just HugeTLB. I decoupled this as a
> parameter to prevent potential regressions down the road; if a developer leverages
> this for other huge page types in the future, they won't inadvertently break
> things by forgetting to update a hardcoded false in init_pageblock_migratetype(),
> especially since memmap_init_range() natively accepts an isolate parameter.
> 
> Of course, we could also just delete this parameter for now and add it back later if
> needed. I think both approaches work.
> 
> Which way are you leaning?

I'd drop it for now and would revisit when there would be a new HVO user. 
Even one boolean means it's hard to tell from a call site what is the
intention of the flag, two make it completely confusing :)
 
> >> {
> >> 	const unsigned long end = pfn + nr_pages;
> >> 
> >> 	for (pfn = pageblock_align(pfn); pfn < end; pfn += pageblock_nr_pages) {
> >> - 		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false);
> >> + 		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, isolate);
> >> +#ifdef CONFIG_SPARSEMEM
> >> 		if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION))
> >> +#else
> >> + 		if (!atomic && IS_ALIGNED(pfn, MAX_FOLIO_NR_PAGES))
> >> +#endif
> > 
> > Let's trigger cond_resched() on some defined number of iterations or some
> > memory size chunk, e.g PAGES_PER_128M or even PAGES_PER_1G.
> 
> Yes, that's for the best. I was really struggling to choose a suitable macro
> for this earlier, but I realized it's a difficult thing to get right. I'm leaning
> toward selecting PAGES_PER_1G instead.

Yeah, PAGES_PER_1G makes sense to me too.
 
> Muchun,
> Thanks.
> 
> > 
> >> 			cond_resched();

-- 
Sincerely yours,
Mike.

Re: [PATCH 04/17] mm/mm_init: skip initializing shared vmemmap tail pages

[Muchun Song]

> On Jul 9, 2026, at 21:05, Mike Rapoport <rppt@kernel.org> wrote:
> 
> On Thu, Jul 09, 2026 at 08:31:31PM +0800, Muchun Song wrote:
>>> On Jul 9, 2026, at 18:45, Mike Rapoport <rppt@kernel.org> wrote:
>>> On Thu, Jul 02, 2026 at 05:38:08PM +0800, Muchun Song wrote:
>>>> memmap_init_range() initializes every struct page in the target range.
>>>> For compound pages with vmemmap optimization, the tail struct pages are
>>>> backed by a shared vmemmap page.
>>>> 
>>>> Initializing those tail struct pages would overwrite the shared
>>>> vmemmap page contents, so users such as HugeTLB have to open-code
>>>> follow-up handling to restore the metadata afterwards.
>>>> 
>>>> Use the section's compound page order to detect struct pages that fall
>>>> into the shared tail vmemmap range and skip their initialization in
>>>> memmap_init_range().  Still initialize the pageblock migratetypes for
>>>> the skipped range so the surrounding setup remains intact.
>>>> 
>>>> This is a preparatory change for consolidating handling across users of
>>>> vmemmap optimization, and it also avoids redundant initialization of
>>>> shared tail vmemmap pages during early boot.
>>>> 
>>>> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>>>> ---
>>>> include/linux/mmzone.h |  4 ++++
>>>> mm/internal.h          | 16 ++++++++++++++++
>>>> mm/mm_init.c           | 25 +++++++++++++++++++------
>>>> 3 files changed, 39 insertions(+), 6 deletions(-)
>>>> 
>>>> @@ -673,19 +673,21 @@ static inline void fixup_hashdist(void)
>>>> static inline void fixup_hashdist(void) {}
>>>> #endif /* CONFIG_NUMA */
>>>> 
>>>> -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
>>>> static __meminit void pageblock_migratetype_init_range(unsigned long pfn,
>>>> - unsigned long nr_pages, int migratetype, bool atomic)
>>>> + unsigned long nr_pages, int migratetype, bool isolate, bool atomic)
>>> 
>>> What is isolate parameter for?
>> 
>> I've re-examined the code, and you're right that the isolate parameter is technically
>> redundant for our current use case, as memmap_init_zone_range() passes false.
>> 
>> The rationale behind keeping it is future-proofing. The ultimate goal of a generic
>> HVO is to support arbitrary huge pages, not just HugeTLB. I decoupled this as a
>> parameter to prevent potential regressions down the road; if a developer leverages
>> this for other huge page types in the future, they won't inadvertently break
>> things by forgetting to update a hardcoded false in init_pageblock_migratetype(),
>> especially since memmap_init_range() natively accepts an isolate parameter.
>> 
>> Of course, we could also just delete this parameter for now and add it back later if
>> needed. I think both approaches work.
>> 
>> Which way are you leaning?
> 
> I'd drop it for now and would revisit when there would be a new HVO user. 
> Even one boolean means it's hard to tell from a call site what is the
> intention of the flag, two make it completely confusing :)

No problem. I'll drop it next version.

Muchun,
Thanks.

> 
>>>> {
>>>> const unsigned long end = pfn + nr_pages;
>>>> 
>>>> for (pfn = pageblock_align(pfn); pfn < end; pfn += pageblock_nr_pages) {
>>>> - init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false);
>>>> + init_pageblock_migratetype(pfn_to_page(pfn), migratetype, isolate);
>>>> +#ifdef CONFIG_SPARSEMEM
>>>> if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION))
>>>> +#else
>>>> + if (!atomic && IS_ALIGNED(pfn, MAX_FOLIO_NR_PAGES))
>>>> +#endif
>>> 
>>> Let's trigger cond_resched() on some defined number of iterations or some
>>> memory size chunk, e.g PAGES_PER_128M or even PAGES_PER_1G.
>> 
>> Yes, that's for the best. I was really struggling to choose a suitable macro
>> for this earlier, but I realized it's a difficult thing to get right. I'm leaning
>> toward selecting PAGES_PER_1G instead.
> 
> Yeah, PAGES_PER_1G makes sense to me too.
> 
>> Muchun,
>> Thanks.
>> 
>>> 
>>>> cond_resched();
> 
> -- 
> Sincerely yours,
> Mike.

Re: [PATCH 04/17] mm/mm_init: skip initializing shared vmemmap tail pages

[Mike Rapoport]

> memmap_init_range() initializes every struct page in the target range.
> For compound pages with vmemmap optimization, the tail struct pages are
> backed by a shared vmemmap page.
> 
> Initializing those tail struct pages would overwrite the shared
> vmemmap page contents, so users such as HugeTLB have to open-code
> follow-up handling to restore the metadata afterwards.
> 
> Use the section's compound page order to detect struct pages that fall
> into the shared tail vmemmap range and skip their initialization in
> memmap_init_range().  Still initialize the pageblock migratetypes for
> the skipped range so the surrounding setup remains intact.
> 
> This is a preparatory change for consolidating handling across users of
> vmemmap optimization, and it also avoids redundant initialization of
> shared tail vmemmap pages during early boot.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index ea884245f499..6fa6e7f0abf9 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -2375,6 +2375,10 @@ struct mem_section;
>  
>  #define sparse_vmemmap_init_nid_early(_nid) do {} while (0)
>  #define pfn_in_present_section pfn_valid
> +static inline struct mem_section *__pfn_to_section(unsigned long pfn)
> +{
> +	return NULL;
> +}

Changelog should mention why this is a part of the patch, it's really
not obvious.

>  #endif /* CONFIG_SPARSEMEM */
>  
>  #ifdef CONFIG_SPARSEMEM_VMEMMAP
> diff --git a/mm/internal.h b/mm/internal.h
> index 430aa72a4575..ebbab7421633 100644
> --- a/mm/internal.h
> +++ b/mm/internal.h
> @@ -1002,10 +1002,26 @@ static inline void sparse_init(void) {}
>   */
>  #ifdef CONFIG_SPARSEMEM_VMEMMAP
>  void sparse_init_subsection_map(void);
> +
> +static inline bool page_vmemmap_optimizable(const struct page *page, unsigned int order)
> +{
> +	const unsigned long pfn = page_to_pfn(page);
> +	const unsigned long nr_pages = 1UL << order;
> +

Don't you want to gate it on HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON?

Even if it's false for sections without order explicitly set (which is
btw not very obvious), this adds checks for every struct page that we
initialize.

We can revisit this later when the optimization would be relevant for
memory hotplug.

> +	if (!is_power_of_2(sizeof(struct page)))
> +		return false;
> +
> +	return (pfn & (nr_pages - 1)) >= OPTIMIZED_FOLIO_VMEMMAP_NR_STRUCT_PAGES;
> +}
>  #else
>  static inline void sparse_init_subsection_map(void)
>  {
>  }
> +
> +static inline bool page_vmemmap_optimizable(const struct page *page, unsigned int order)
> +{
> +	return false;
> +}
>  #endif /* CONFIG_SPARSEMEM_VMEMMAP */
>  
>  #if defined CONFIG_COMPACTION || defined CONFIG_CMA
> diff --git a/mm/mm_init.c b/mm/mm_init.c
> index 4026b084bd4b..7ef1ac105058 100644
> --- a/mm/mm_init.c
> +++ b/mm/mm_init.c
> @@ -674,19 +674,21 @@ static inline void fixup_hashdist(void)
>  static inline void fixup_hashdist(void) {}
>  #endif /* CONFIG_NUMA */
>  
> -#if defined(CONFIG_ZONE_DEVICE) || defined(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
>  static __meminit void pageblock_migratetype_init_range(unsigned long pfn,
> -		unsigned long nr_pages, int migratetype, bool atomic)
> +		unsigned long nr_pages, int migratetype, bool isolate, bool atomic)
>  {
>  	const unsigned long end = pfn + nr_pages;
>  
>  	for (pfn = pageblock_align(pfn); pfn < end; pfn += pageblock_nr_pages) {
> -		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, false);
> +		init_pageblock_migratetype(pfn_to_page(pfn), migratetype, isolate);
> +#ifdef CONFIG_SPARSEMEM
>  		if (!atomic && IS_ALIGNED(pfn, PAGES_PER_SECTION))
> +#else
> +		if (!atomic && IS_ALIGNED(pfn, MAX_FOLIO_NR_PAGES))
> +#endif
>  			cond_resched();
>  	}
>  }
> -#endif
>  
>  #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
>  static inline void pgdat_set_deferred_range(pg_data_t *pgdat)
> @@ -892,6 +894,8 @@ void __meminit memmap_init_range(unsigned long size, int nid, unsigned long zone
>  #endif
>  
>  	for (pfn = start_pfn; pfn < end_pfn; ) {
> +		unsigned int order = section_order(__pfn_to_section(pfn));
> +

I'd move this into page_vmemmap_optimizable(). This way it would be
clearer that a section must have non 0 order for optimization to catch.

>  		/*
>  		 * There can be holes in boot-time mem_map[]s handed to this
>  		 * function.  They do not exist on hotplugged memory.
> @@ -906,6 +910,15 @@ void __meminit memmap_init_range(unsigned long size, int nid, unsigned long zone
>  		}
>  
>  		page = pfn_to_page(pfn);
> +		if (page_vmemmap_optimizable(page, order)) {

You can pass pfn instead of page to page_vmemmap_optimizable(), will
save a conversion there.

> +			const unsigned long start = pfn;
> +
> +			pfn = min(ALIGN(start, 1UL << order), end_pfn);
> +			pageblock_migratetype_init_range(start, pfn - start, migratetype,
> +							 isolate_pageblock, false);

I wonder if we'll see any measurable difference in the memmap initialization
if we pull pageblock initialization out unconditionally.

If we don't we can just call pageblock_migratetype_init_range() for the
entire range and kill that

		if (pageblock_aligned(pfn)) {
			init_pageblock_migratetype(page, migratetype,
					isolate_pageblock);
			cond_resched();
		}

-- 
Sincerely yours,
Mike.

Re: [PATCH 06/17] mm/sparse-vmemmap: support section-based vmemmap accounting

[Mike Rapoport]

> section_nr_vmemmap_pages() is used to account the vmemmap pages consumed by a
> memory section, but it currently only understands the ordinary case and the
> pgmap-provided optimization case. That is not enough for section-based vmemmap
> optimization, where the compound page order is carried by the memory section
> itself and tail vmemmap pages may be shared.
> 
> Make the helper report the actual vmemmap footprint of a section, so it can be
> used as the common accounting path for both ordinary and optimized sections.
> 
> Teach section_nr_vmemmap_pages() to use the section order when there is no
> pgmap, and to account only the vmemmap pages that are actually needed for an
> optimizable section. This keeps the accounting consistent with section-based
> vmemmap optimization.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 6fa6e7f0abf9..41a1ebbe5e85 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -2403,6 +2403,14 @@ static inline unsigned int section_order(const struct mem_section *section)
>  }
>  #endif
>  
> +static inline bool section_vmemmap_optimizable(const struct mem_section *section)
> +{
> +	if (!is_power_of_2(sizeof(struct page)))
> +		return false;
> +
> +	return section_order(section) >= OPTIMIZABLE_FOLIO_MIN_ORDER;

Like with page_vmemmap_optimizable() I think it's not obvious that
section_order() is enough to gate vmemmap optimization.

> +}
> +
>  /*
>   * Fallback case for when the architecture provides its own pfn_valid() but
>   * not a corresponding for_each_valid_pfn().
> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
> index 77154e9da08c..99f59394feab 100644
> --- a/mm/sparse-vmemmap.c
> +++ b/mm/sparse-vmemmap.c
> @@ -645,24 +645,29 @@ void offline_mem_sections(unsigned long start_pfn, unsigned long end_pfn)
>  static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages,
>  		struct vmem_altmap *altmap, struct dev_pagemap *pgmap)
>  {
> -	const unsigned int order = pgmap ? pgmap->vmemmap_shift : 0;
> +	const struct mem_section *ms = __pfn_to_section(pfn);
> +	const unsigned int order = pgmap ? pgmap->vmemmap_shift : section_order(ms);
>  	const unsigned long pages_per_compound = 1UL << order;
> +	unsigned int vmemmap_pages = OPTIMIZED_FOLIO_VMEMMAP_PAGES;
>  
>  	VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SUBSECTION));
>  	VM_WARN_ON_ONCE(nr_pages > PAGES_PER_SECTION);
>  
> -	if (!vmemmap_can_optimize(altmap, pgmap))
> +	if (vmemmap_can_optimize(altmap, pgmap))
> +		vmemmap_pages = VMEMMAP_RESERVE_NR;

I'd put it below the next if ...

> +
> +	if (!vmemmap_can_optimize(altmap, pgmap) && !section_vmemmap_optimizable(ms))
>  		return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE);

... and make it

	vmmemap_pages = vmemmap_can_optimize() ? VMEMMAP_RESERVE_NR : 1;

-- 
Sincerely yours,
Mike.

Re: [PATCH 06/17] mm/sparse-vmemmap: support section-based vmemmap accounting

[Muchun Song]

> On Jul 15, 2026, at 13:08, Mike Rapoport <rppt@kernel.org> wrote:
> 
>> section_nr_vmemmap_pages() is used to account the vmemmap pages consumed by a
>> memory section, but it currently only understands the ordinary case and the
>> pgmap-provided optimization case. That is not enough for section-based vmemmap
>> optimization, where the compound page order is carried by the memory section
>> itself and tail vmemmap pages may be shared.
>> 
>> Make the helper report the actual vmemmap footprint of a section, so it can be
>> used as the common accounting path for both ordinary and optimized sections.
>> 
>> Teach section_nr_vmemmap_pages() to use the section order when there is no
>> pgmap, and to account only the vmemmap pages that are actually needed for an
>> optimizable section. This keeps the accounting consistent with section-based
>> vmemmap optimization.
>> 
>> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>> 
>> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
>> index 6fa6e7f0abf9..41a1ebbe5e85 100644
>> --- a/include/linux/mmzone.h
>> +++ b/include/linux/mmzone.h
>> @@ -2403,6 +2403,14 @@ static inline unsigned int section_order(const struct mem_section *section)
>> }
>> #endif
>> 
>> +static inline bool section_vmemmap_optimizable(const struct mem_section *section)
>> +{
>> + 	if (!is_power_of_2(sizeof(struct page)))
>> + 		return false;
>> +
>> + 	return section_order(section) >= OPTIMIZABLE_FOLIO_MIN_ORDER;
> 
> Like with page_vmemmap_optimizable() I think it's not obvious that
> section_order() is enough to gate vmemmap optimization.

page_vmemmap_optimizable() checks the optimizability of an individual page.
section_vmemmap_optimizable() checks whether the optimization can be applied
to a section. Some people may think that section_vmemmap_optimizable() checks
the optimizability of all pages within an entire section. You are referring
to that ambiguity, right?

What do you think about renaming it to section_vmemmap_optimization_eligible()?
Would that clear up the ambiguity? Or should we add a comment to
section_vmemmap_optimizableto explain its purpose?

> 
>> +}
>> +
>> /*
>>  * Fallback case for when the architecture provides its own pfn_valid() but
>>  * not a corresponding for_each_valid_pfn().
>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
>> index 77154e9da08c..99f59394feab 100644
>> --- a/mm/sparse-vmemmap.c
>> +++ b/mm/sparse-vmemmap.c
>> @@ -645,24 +645,29 @@ void offline_mem_sections(unsigned long start_pfn, unsigned long end_pfn)
>> static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages,
>> struct vmem_altmap *altmap, struct dev_pagemap *pgmap)
>> {
>> - 	const unsigned int order = pgmap ? pgmap->vmemmap_shift : 0;
>> + 	const struct mem_section *ms = __pfn_to_section(pfn);
>> + 	const unsigned int order = pgmap ? pgmap->vmemmap_shift : section_order(ms);
>> 	const unsigned long pages_per_compound = 1UL << order;
>> + 	unsigned int vmemmap_pages = OPTIMIZED_FOLIO_VMEMMAP_PAGES;
>> 
>> 	VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SUBSECTION));
>> 	VM_WARN_ON_ONCE(nr_pages > PAGES_PER_SECTION);
>> 
>> - 	if (!vmemmap_can_optimize(altmap, pgmap))
>> + 	if (vmemmap_can_optimize(altmap, pgmap))
>> + 		vmemmap_pages = VMEMMAP_RESERVE_NR;
> 
> I'd put it below the next if ...

Make sense. Will do.

Thanks for your review.

Muchun

> 
>> +
>> + 	if (!vmemmap_can_optimize(altmap, pgmap) && !section_vmemmap_optimizable(ms))
>> 		return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE);
> 
> ... and make it
> 
> vmmemap_pages = vmemmap_can_optimize() ? VMEMMAP_RESERVE_NR : 1;
> 
> -- 
> Sincerely yours,
> Mike.
> 

Re: [PATCH 07/17] mm/sparse-vmemmap: support section-based vmemmap optimization

[Mike Rapoport]

> Teach sparse-vmemmap population code to use the compound page order
> when deciding whether a vmemmap page can be optimized.
> 
> With this information, the common sparse-vmemmap population path can
> allocate or reuse shared tail vmemmap pages directly instead of relying
> on HugeTLB/DAX-specific handling.
> 
> This centralizes vmemmap optimization logic in the sparse-vmemmap code,
> based on section metadata, and prepares for sharing the same mechanism
> across different users of vmemmap optimization, including HugeTLB and
> DAX.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index 41a1ebbe5e85..41f6a584a52a 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -1148,7 +1148,7 @@ struct zone {
>  	/* Zone statistics */
>  	atomic_long_t		vm_stat[NR_VM_ZONE_STAT_ITEMS];
>  	atomic_long_t		vm_numa_event[NR_VM_NUMA_EVENT_ITEMS];
> -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
>  	struct page *vmemmap_tails[NR_OPTIMIZABLE_FOLIO_ORDERS];
>  #endif
>  } ____cacheline_internodealigned_in_smp;
> diff --git a/mm/internal.h b/mm/internal.h
> index ebbab7421633..8ec8b447d75e 100644
> --- a/mm/internal.h
> +++ b/mm/internal.h
> @@ -993,6 +993,9 @@ static inline void __section_mark_present(struct mem_section *ms,
>  
>  	ms->section_mem_map |= SECTION_MARKED_PRESENT;
>  }
> +
> +int section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages,
> +		struct vmem_altmap *altmap, struct dev_pagemap *pgmap);

What about !SPARSE_VMEMMAP case?

>  #else
>  static inline void sparse_init(void) {}
>  #endif /* CONFIG_SPARSEMEM */
> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
> index 99f59394feab..232f2a08afad 100644
> --- a/mm/sparse-vmemmap.c
> +++ b/mm/sparse-vmemmap.c
> @@ -146,17 +146,49 @@ void __meminit vmemmap_verify(pte_t *pte, int node,
>  			start, end - 1);
>  }
>  
> +static struct zone __meminit *pfn_to_zone(unsigned long pfn, int nid)
> +{
> +	pg_data_t *pgdat = NODE_DATA(nid);
> +
> +	for (enum zone_type zone_type = 0; zone_type < MAX_NR_ZONES; zone_type++) {
> +		struct zone *zone = &pgdat->node_zones[zone_type];
> +
> +		if (zone_spans_pfn(zone, pfn))
> +			return zone;
> +	}
> +
> +	return NULL;
> +}
> +
> +static __meminit struct page *vmemmap_get_tail(unsigned int order, struct zone *zone);
> +

Maybe just move the entire function here?

>  static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
>  				       struct vmem_altmap *altmap,
>  				       unsigned long ptpfn, unsigned long flags)
>  {
>  	pte_t *pte = pte_offset_kernel(pmd, addr);
> +	struct page *page = (struct page *)addr;
> +	unsigned long pfn = page_to_pfn(page);
> +	unsigned int order = section_order(__pfn_to_section(pfn));
> +
>  	if (pte_none(ptep_get(pte))) {
>  		pte_t entry;
> -		void *p;
> +
> +		if (page_vmemmap_optimizable(page, order) &&
> +		    ptpfn == (unsigned long)-1) {
> +			struct zone *zone = pfn_to_zone(pfn, node);
> +
> +			if (WARN_ON_ONCE(!zone))
> +				return NULL;
> +			page = vmemmap_get_tail(order, zone);
> +			if (!page)
> +				return NULL;
> +			ptpfn = page_to_pfn(page);
> +		}
>  
>  		if (ptpfn == (unsigned long)-1) {
> -			p = vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap);
> +			void *p = vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap);
> +
>  			if (!p)
>  				return NULL;
>  			ptpfn = PHYS_PFN(__pa(p));
> @@ -175,7 +207,8 @@ static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, in
>  		}
>  		entry = pfn_pte(ptpfn, PAGE_KERNEL);
>  		set_pte_at(&init_mm, addr, pte, entry);
> -	}
> +	} else if (WARN_ON_ONCE(page_vmemmap_optimizable(page, order)))
> +		return NULL;
>  	return pte;
>  }
>  
> @@ -320,7 +353,6 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigned long end,
>  	}
>  }
>  
> -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
>  static __meminit struct page *vmemmap_get_tail(unsigned int order, struct zone *zone)
>  {
>  	struct page *p, *tail;
> @@ -348,6 +380,7 @@ static __meminit struct page *vmemmap_get_tail(unsigned int order, struct zone *
>  	return tail;
>  }
>  
> +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
>  int __meminit vmemmap_populate_hvo(unsigned long addr, unsigned long end,
>  				       unsigned int order, struct zone *zone,
>  				       unsigned long headsize)
> @@ -402,6 +435,9 @@ int __meminit vmemmap_populate_hugepages(unsigned long start, unsigned long end,
>  	pmd_t *pmd;
>  
>  	for (addr = start; addr < end; addr = next) {
> +		unsigned long pfn = page_to_pfn((struct page *)addr);
> +		struct mem_section *ms = __pfn_to_section(pfn);
> +
>  		next = pmd_addr_end(addr, end);
>  
>  		pgd = vmemmap_pgd_populate(addr, node);
> @@ -417,7 +453,7 @@ int __meminit vmemmap_populate_hugepages(unsigned long start, unsigned long end,
>  			return -ENOMEM;
>  
>  		pmd = pmd_offset(pud, addr);
> -		if (pmd_none(pmdp_get(pmd))) {
> +		if (pmd_none(pmdp_get(pmd)) && !section_vmemmap_optimizable(ms)) {
>  			void *p;
>  
>  			p = vmemmap_alloc_block_buf(PMD_SIZE, node, altmap);
> @@ -435,8 +471,19 @@ int __meminit vmemmap_populate_hugepages(unsigned long start, unsigned long end,
>  				 */
>  				return -ENOMEM;
>  			}
> -		} else if (vmemmap_check_pmd(pmd, node, addr, next))
> +		} else if (vmemmap_check_pmd(pmd, node, addr, next)) {
> +			const struct mem_section *start_ms;
> +			unsigned long align = max(1UL << section_order(ms), PAGES_PER_SECTION);
> +
> +			/* HVO-covered sections must not use PMD mappings. */
> +			start_ms = __pfn_to_section(ALIGN_DOWN(pfn, align));
> +			if (!IS_ALIGNED(pfn, align) && section_vmemmap_optimizable(start_ms))
> +				return -ENOTSUPP;
> +
> +			/* PMD mappings end HVO coverage for this section. */
> +			section_set_order(ms, 0);

The whole pmd block here becomes really hairy. 
I'd suggest splitting it out to a helper function and reducing the
amount of nested ifs with early returns where possible.

>  			continue;
> +		}
>  		if (vmemmap_populate_basepages(addr, next, node, altmap))
>  			return -ENOMEM;
>  	}
> @@ -642,36 +689,6 @@ void offline_mem_sections(unsigned long start_pfn, unsigned long end_pfn)
>  	}
>  }
>  
> -static int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages,
> -		struct vmem_altmap *altmap, struct dev_pagemap *pgmap)
> -{
> -	const struct mem_section *ms = __pfn_to_section(pfn);
> -	const unsigned int order = pgmap ? pgmap->vmemmap_shift : section_order(ms);
> -	const unsigned long pages_per_compound = 1UL << order;
> -	unsigned int vmemmap_pages = OPTIMIZED_FOLIO_VMEMMAP_PAGES;
> -
> -	VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SUBSECTION));
> -	VM_WARN_ON_ONCE(nr_pages > PAGES_PER_SECTION);
> -
> -	if (vmemmap_can_optimize(altmap, pgmap))
> -		vmemmap_pages = VMEMMAP_RESERVE_NR;
> -
> -	if (!vmemmap_can_optimize(altmap, pgmap) && !section_vmemmap_optimizable(ms))
> -		return DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE);
> -
> -	if (order < PFN_SECTION_SHIFT) {
> -		VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, pages_per_compound));
> -		return vmemmap_pages * nr_pages / pages_per_compound;
> -	}
> -
> -	VM_WARN_ON_ONCE(!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION));
> -
> -	if (IS_ALIGNED(pfn, pages_per_compound))
> -		return vmemmap_pages;
> -
> -	return 0;
> -}
> -
>  static struct page * __meminit populate_section_memmap(unsigned long pfn,
>  		unsigned long nr_pages, int nid, struct vmem_altmap *altmap,
>  		struct dev_pagemap *pgmap)
> diff --git a/mm/sparse.c b/mm/sparse.c
> index 9457a4d6a6fc..3e96478a63e0 100644
> --- a/mm/sparse.c
> +++ b/mm/sparse.c
> @@ -284,6 +284,36 @@ static void __init sparse_usage_fini(void)
>  	sparse_usagebuf = sparse_usagebuf_end = NULL;
>  }
>  
> +int __meminit section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages,
> +		struct vmem_altmap *altmap, struct dev_pagemap *pgmap)
> +{

I don't think moving this to sparse.c is a good idea, let's keep vmemmap
code in sparse-vmemmap.c

-- 
Sincerely yours,
Mike.

Re: [PATCH 03/17] mm/sparse-vmemmap: introduce folio-oriented vmemmap optimization macros

[Mike Rapoport]

> Introduce macros for those folio-oriented properties and use them to
> replace the tail-based names in the HVO paths. This makes the code read
> in terms of folio semantics instead of an implementation detail and
> better matches other folio-based users such as DAX.
> 
> No functional change intended.
> 
> Signed-off-by: Muchun Song <songmuchun@bytedance.com>
>
> diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> index bacd89572c5c..ea884245f499 100644
> --- a/include/linux/mmzone.h
> +++ b/include/linux/mmzone.h
> @@ -107,13 +107,15 @@
>  	 is_power_of_2(sizeof(struct page)) ? \
>  	 MAX_FOLIO_NR_PAGES * sizeof(struct page) : 0)
>  
> -/*
> - * vmemmap optimization (like HVO) is only possible for page orders that fill
> - * two or more pages with struct pages.
> - */
> -#define VMEMMAP_TAIL_MIN_ORDER (ilog2(2 * PAGE_SIZE / sizeof(struct page)))
> -#define __NR_VMEMMAP_TAILS (MAX_FOLIO_ORDER - VMEMMAP_TAIL_MIN_ORDER + 1)
> -#define NR_VMEMMAP_TAILS (__NR_VMEMMAP_TAILS > 0 ? __NR_VMEMMAP_TAILS : 0)
> +/* The number of vmemmap pages required by a vmemmap-optimized folio. */
> +#define OPTIMIZED_FOLIO_VMEMMAP_PAGES		1
> +#define OPTIMIZED_FOLIO_VMEMMAP_SIZE		(OPTIMIZED_FOLIO_VMEMMAP_PAGES * PAGE_SIZE)
> +#define OPTIMIZED_FOLIO_VMEMMAP_NR_STRUCT_PAGES	(OPTIMIZED_FOLIO_VMEMMAP_SIZE / sizeof(struct page))
> +#define OPTIMIZABLE_FOLIO_MIN_ORDER		(ilog2(OPTIMIZED_FOLIO_VMEMMAP_NR_STRUCT_PAGES) + 1)
> +
> +#define __NR_OPTIMIZABLE_FOLIO_ORDERS		(MAX_FOLIO_ORDER - OPTIMIZABLE_FOLIO_MIN_ORDER + 1)
> +#define NR_OPTIMIZABLE_FOLIO_ORDERS		\
> +	(__NR_OPTIMIZABLE_FOLIO_ORDERS > 0 ? __NR_OPTIMIZABLE_FOLIO_ORDERS : 0)

I can't say I like the new names, they read like some folio optimization
rather than vmemmap optimization.

I think they should be namespaced with VMEMMAP_ or even
VMEMMAP_OPTIMIZATION_ (or VMEMMAP_OPT_) and should not mention folio.

I'd also drop intermediate defines OPTIMIZED_FOLIO_VMEMMAP_PAGES and
OPTIMIZED_FOLIO_VMEMMAP_SIZE and use 1 and PAGE_SIZE directly.

-- 
Sincerely yours,
Mike.

Re: [PATCH 01/17] mm/sparse: drop power-of-2 size requirement for struct mem_section

[Mike Rapoport]

On Thu, 02 Jul 2026 17:38:05 +0800, Muchun Song <songmuchun@bytedance.com> wrote:
> struct mem_section is currently forced to a power-of-2 size so the
> section-to-root lookup can use a mask instead of a modulo.
> 
> That requirement adds configuration-dependent padding, especially with
> CONFIG_PAGE_EXTENSION, just to preserve the lookup scheme.
> 
> Drop the constraint and use a plain modulo for the lookup instead. The
> divisor is constant, so the generated code remains cheap while avoiding
> the extra padding. It also removes an unnecessary layout constraint
> from the type.
> 
> [...]

Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>

-- 
Sincerely yours,
Mike.

Re: [PATCH 01/17] mm/sparse: drop power-of-2 size requirement for struct mem_section

[Muchun Song]

> On Jul 15, 2026, at 17:34, David Laight <david.laight.linux@gmail.com> wrote:
> 
> On Thu,  2 Jul 2026 17:38:05 +0800
> Muchun Song <songmuchun@bytedance.com> wrote:
> 
>> struct mem_section is currently forced to a power-of-2 size so the
>> section-to-root lookup can use a mask instead of a modulo.
>> 
>> That requirement adds configuration-dependent padding, especially with
>> CONFIG_PAGE_EXTENSION, just to preserve the lookup scheme.
>> 
>> Drop the constraint and use a plain modulo for the lookup instead. The
>> divisor is constant, so the generated code remains cheap while avoiding
>> the extra padding. It also removes an unnecessary layout constraint
>> from the type.
> 
> This has a side effect of changing the size of the 'section' from
> PAGE_SIZE to something 'a bit smaller' when CONFIG_PAGE_EXTENSION
> is defined.
> I don't think it actually matters, the allocation is done by:
> 
> static noinline struct mem_section __ref *sparse_index_alloc(int nid)
> {
> 	struct mem_section *section = NULL;
> 	unsigned long array_size = SECTIONS_PER_ROOT *
> 				   sizeof(struct mem_section);
> 
> 	if (slab_is_available()) {
> 		section = kzalloc_node(array_size, GFP_KERNEL, nid);
> 	} else {
> 		section = memblock_alloc_node(array_size, SMP_CACHE_BYTES,
> 					      nid);
> 
> so the size might get rounded up to PAGE_SIZE anyway.

I'm not sure I really understand what you mean. You might be asking whether
the reduction in the size of the `mem_section` structure does not actually result
in memory savings? If so, please let me explain clearly. As you mentioned, the
size of memory allocated each time here should be PAGE_SIZE. Before the
modification, one page could hold 4096/32 = 128 `struct mem_section` instances;
with the modified code, the number of `struct mem_section` instances that can fit
is 4096/24 = 170. Therefore, the range of memory sections that a PAGE_SIZE can
cover has increased 32%.

Please let me know if I didn’t get your point.

> 
> I also suspect that '% 24u' might be enough slower than '% 32u' to
> generate a measurable performance drop.
> (It doesn't matter whether you do '& 31' or '% 32u'.)

David, I agree that % 24u is slower than % 32u — the latter maps to ‘and', while the
former requires a multiply-shift sequence. However, since the divisor is a constant,
the compiler should uses the magic multiplier approach at -O2, which is ~3-5 cycles
instead of 1. So I think the per-lookup overhead is real but small.

Thanks for your review.

Muchun

> 
> 	David

Re: [PATCH 01/17] mm/sparse: drop power-of-2 size requirement for struct mem_section

[David Laight]

On Wed, 15 Jul 2026 21:15:45 +0800
Muchun Song <muchun.song@linux.dev> wrote:

> > On Jul 15, 2026, at 17:34, David Laight <david.laight.linux@gmail.com> wrote:
> > 
> > On Thu,  2 Jul 2026 17:38:05 +0800
> > Muchun Song <songmuchun@bytedance.com> wrote:
> >   
> >> struct mem_section is currently forced to a power-of-2 size so the
> >> section-to-root lookup can use a mask instead of a modulo.
> >> 
> >> That requirement adds configuration-dependent padding, especially with
> >> CONFIG_PAGE_EXTENSION, just to preserve the lookup scheme.
> >> 
> >> Drop the constraint and use a plain modulo for the lookup instead. The
> >> divisor is constant, so the generated code remains cheap while avoiding
> >> the extra padding. It also removes an unnecessary layout constraint
> >> from the type.  
> > 
> > This has a side effect of changing the size of the 'section' from
> > PAGE_SIZE to something 'a bit smaller' when CONFIG_PAGE_EXTENSION
> > is defined.
> > I don't think it actually matters, the allocation is done by:
> > 
> > static noinline struct mem_section __ref *sparse_index_alloc(int nid)
> > {
> > 	struct mem_section *section = NULL;
> > 	unsigned long array_size = SECTIONS_PER_ROOT *
> > 				   sizeof(struct mem_section);
> > 
> > 	if (slab_is_available()) {
> > 		section = kzalloc_node(array_size, GFP_KERNEL, nid);
> > 	} else {
> > 		section = memblock_alloc_node(array_size, SMP_CACHE_BYTES,
> > 					      nid);
> > 
> > so the size might get rounded up to PAGE_SIZE anyway.  
> 
> I'm not sure I really understand what you mean. You might be asking whether
> the reduction in the size of the `mem_section` structure does not actually result
> in memory savings? If so, please let me explain clearly. As you mentioned, the
> size of memory allocated each time here should be PAGE_SIZE. Before the
> modification, one page could hold 4096/32 = 128 `struct mem_section` instances;
> with the modified code, the number of `struct mem_section` instances that can fit
> is 4096/24 = 170. Therefore, the range of memory sections that a PAGE_SIZE can
> cover has increased 32%.
> 
> Please let me know if I didn’t get your point.

170 * 24 is 4080 - 16 bytes less than PAGE_SIZE.
In principle kmalloc() need not allocate a full page for it which would
lead to the data crossing a page boundary - which may not be intended.

> 
> > 
> > I also suspect that '% 24u' might be enough slower than '% 32u' to
> > generate a measurable performance drop.
> > (It doesn't matter whether you do '& 31' or '% 32u'.)  
> 
> David, I agree that % 24u is slower than % 32u — the latter maps to ‘and', while the
> former requires a multiply-shift sequence. However, since the divisor is a constant,
> the compiler should uses the magic multiplier approach at -O2, which is ~3-5 cycles
> instead of 1. So I think the per-lookup overhead is real but small.

The divide/remainder is by 170 not 24 - but the effect is the same.

Indeed, but it might be a hot enough path to be measurable.
I've just done a pile of test compiles, see https://godbolt.org/z/zq8WjaqGj

The remainder code has to do the divide, gcc might notice it has already
done it - but it would be better to do it explicitly.
However, both sparc64 and looooongarch64 end up executing divides.
Also some of the others aren't as short as you might expect, both
generating the constant for the multiply and avoiding a 'mul' instruction
for the remainder (esp. s390).

	David

> 
> Thanks for your review.
> 
> Muchun
> 
> > 
> > 	David  
>