[PATCH 0/3] Subject: mm/hugetlb: fix CMA handling and surplus allocation accounting

[PATCH 0/3] Subject: mm/hugetlb: fix CMA handling and surplus allocation accounting

[Kaitao Cheng]

From: Kaitao Cheng <chengkaitao@kylinos.cn>

This series fixes three independent HugeTLB issues reported by Sashiko,
covering CMA parameter parsing, CMA allocation state preservation, and
NR_HUGETLB accounting on allocation rollback.

Patch 1 rejects a global hugetlb_cma size following node-specific entries.
This prevents malformed input from overwriting valid per-node reservations
with an incorrectly parsed global size.

Patch 2 preserves the CMA flag when a folio is returned to the HugeTLB
pool after vmemmap restoration fails. This ensures that a subsequent free
releases the CMA allocation correctly instead of leaving its bitmap set.

Patch 3 fixes NR_HUGETLB accounting when unused surplus folios are freed
during allocation or reservation cleanup.

Kaitao Cheng (3):
  mm/hugetlb_cma: fix incorrect hugetlb_cma_size assignment on malformed
    input
  mm/hugetlb: preserve CMA flag when returning folios to the pool
  mm/hugetlb: fix NR_HUGETLB accounting for unused surplus folios

 mm/hugetlb.c     | 13 +++++++++++--
 mm/hugetlb_cma.c |  3 +++
 2 files changed, 14 insertions(+), 2 deletions(-)

-- 
2.53.0

[PATCH 1/3] mm/hugetlb_cma: fix incorrect hugetlb_cma_size assignment on malformed input

[Kaitao Cheng]

From: Kaitao Cheng <chengkaitao@kylinos.cn>

When hugetlb_cma contains a global size after node-specific entries,
cmdline_parse_hugetlb_cma() enters the global branch and reparses the
original parameter string. For hugetlb_cma=1:2G,4G, this interprets the
initial node ID as a one-byte global size and clears the 2 GiB reservation
recorded for node 1.

Global and node-specific formats cannot be mixed. Return -EINVAL when a
global entry follows a node-specific entry, before updating any settings.
This preserves the valid prefix, so the example retains node 1's 2 GiB
reservation and reports the malformed option.

Fixes: 38e719ab2673 ("hugetlb: support node specified when using cma for gigantic hugepages")
Closes: https://sashiko.dev/#/bug/linux-10f40d3e-3feb-4622-9da8-1fedecb2d0e2
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
 mm/hugetlb_cma.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index 95fd2d190f0d..8e6b20341fc4 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -167,6 +167,9 @@ static int __init cmdline_parse_hugetlb_cma(char *p)
 		} else {
 			char *next;
 
+			if (s != p)
+				return -EINVAL;
+
 			tmp = memparse(p, &next);
 			if (*next == '%') {
 				if (tmp > 100) {
-- 
2.53.0

Re: [PATCH 1/3] mm/hugetlb_cma: fix incorrect hugetlb_cma_size assignment on malformed input

[Muchun Song]

> On Oct 28, 2026, at 19:33, Kaitao Cheng <kaitao.cheng@linux.dev> wrote:
> 
> From: Kaitao Cheng <chengkaitao@kylinos.cn>
> 
> When hugetlb_cma contains a global size after node-specific entries,
> cmdline_parse_hugetlb_cma() enters the global branch and reparses the
> original parameter string. For hugetlb_cma=1:2G,4G, this interprets the
> initial node ID as a one-byte global size and clears the 2 GiB reservation
> recorded for node 1.
> 
> Global and node-specific formats cannot be mixed. Return -EINVAL when a
> global entry follows a node-specific entry, before updating any settings.
> This preserves the valid prefix, so the example retains node 1's 2 GiB
> reservation and reports the malformed option.
> 
> Fixes: 38e719ab2673 ("hugetlb: support node specified when using cma for gigantic hugepages")
> Closes: https://sashiko.dev/#/bug/linux-10f40d3e-3feb-4622-9da8-1fedecb2d0e2
> Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>


Acked-by: Muchun Song <muchun.song@linux.dev>

Thanks

Re: [PATCH 1/3] mm/hugetlb_cma: fix incorrect hugetlb_cma_size assignment on malformed input

[Oscar Salvador (SUSE)]

On Wed, Oct 28, 2026 at 07:33:17PM +0800, Kaitao Cheng wrote:
> From: Kaitao Cheng <chengkaitao@kylinos.cn>
> 
> When hugetlb_cma contains a global size after node-specific entries,
> cmdline_parse_hugetlb_cma() enters the global branch and reparses the
> original parameter string. For hugetlb_cma=1:2G,4G, this interprets the
> initial node ID as a one-byte global size and clears the 2 GiB reservation
> recorded for node 1.
> 
> Global and node-specific formats cannot be mixed. Return -EINVAL when a
> global entry follows a node-specific entry, before updating any settings.
> This preserves the valid prefix, so the example retains node 1's 2 GiB
> reservation and reports the malformed option.
> 
> Fixes: 38e719ab2673 ("hugetlb: support node specified when using cma for gigantic hugepages")
> Closes: https://sashiko.dev/#/bug/linux-10f40d3e-3feb-4622-9da8-1fedecb2d0e2
> Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>

Reviewed-by: Oscar Salvador <osalvador@suse.de>

> ---
>  mm/hugetlb_cma.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
> index 95fd2d190f0d..8e6b20341fc4 100644
> --- a/mm/hugetlb_cma.c
> +++ b/mm/hugetlb_cma.c
> @@ -167,6 +167,9 @@ static int __init cmdline_parse_hugetlb_cma(char *p)
>  		} else {
>  			char *next;
>  
> +			if (s != p)
> +				return -EINVAL;
> +
>  			tmp = memparse(p, &next);
>  			if (*next == '%') {
>  				if (tmp > 100) {
> -- 
> 2.53.0
> 

-- 
Oscar Salvador
SUSE Labs

[PATCH 2/3] mm/hugetlb: preserve CMA flag when returning folios to the pool

[Kaitao Cheng]

From: Kaitao Cheng <chengkaitao@kylinos.cn>

When vmemmap restoration fails, add_hugetlb_folio() returns the folio
to the hugetlb pool. It clears folio->private and restores only
HPG_vmemmap_optimized, losing HPG_cma on CMA-allocated folios.

A later successful free then bypasses hugetlb_cma_free_frozen_folio()
and returns the pages through free_frozen_pages(). The CMA allocation
bitmap remains set, so the range is unavailable for future CMA
allocations.

Save the CMA flag before resetting folio->private and restore it along
with the vmemmap optimization flag.

Fixes: d2d786714080 ("mm/hugetlb: enable bootmem allocation from CMA areas")
Closes: https://sashiko.dev/#/bug/linux-5363eb09-9c3c-4f8b-9f3c-a998a8c4b258
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
 mm/hugetlb.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 4a008afaedf9..8da4bab676c9 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1437,6 +1437,7 @@ void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
 void add_hugetlb_folio(struct hstate *h, struct folio *folio,
 		       bool adjust_surplus)
 {
+	bool cma = folio_test_hugetlb_cma(folio);
 	int nid = folio_nid(folio);
 
 	VM_BUG_ON_FOLIO(!folio_test_hugetlb_vmemmap_optimized(folio), folio);
@@ -1453,9 +1454,11 @@ void add_hugetlb_folio(struct hstate *h, struct folio *folio,
 	}
 
 	__folio_set_hugetlb(folio);
-	/* Clear all folio->private flags except hugetlb_vmemmap_optimized. */
+	/* Preserve the vmemmap optimization state and CMA allocation origin. */
 	folio->private = NULL;
 	folio_set_hugetlb_vmemmap_optimized(folio);
+	if (cma)
+		folio_set_hugetlb_cma(folio);
 
 	arch_clear_hugetlb_flags(folio);
 	enqueue_hugetlb_folio(h, folio);
-- 
2.53.0

Re: [PATCH 2/3] mm/hugetlb: preserve CMA flag when returning folios to the pool

[Muchun Song]

> On Oct 28, 2026, at 19:33, Kaitao Cheng <kaitao.cheng@linux.dev> wrote:
> 
> From: Kaitao Cheng <chengkaitao@kylinos.cn>
> 
> When vmemmap restoration fails, add_hugetlb_folio() returns the folio
> to the hugetlb pool. It clears folio->private and restores only
> HPG_vmemmap_optimized, losing HPG_cma on CMA-allocated folios.
> 
> A later successful free then bypasses hugetlb_cma_free_frozen_folio()
> and returns the pages through free_frozen_pages(). The CMA allocation
> bitmap remains set, so the range is unavailable for future CMA
> allocations.
> 
> Save the CMA flag before resetting folio->private and restore it along
> with the vmemmap optimization flag.

We already knew this issue. Cc Zi Yan. Seems he hasn't sent a fix.

But we want a different solution. See:

https://lore.kernel.org/linux-mm/A1A14368-7A43-40ED-B2AB-C2EC2C5FFB1D@linux.dev/

Thanks,
Muchun

> 
> Fixes: d2d786714080 ("mm/hugetlb: enable bootmem allocation from CMA areas")
> Closes: https://sashiko.dev/#/bug/linux-5363eb09-9c3c-4f8b-9f3c-a998a8c4b258
> Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
> ---
> mm/hugetlb.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 4a008afaedf9..8da4bab676c9 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1437,6 +1437,7 @@ void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
> void add_hugetlb_folio(struct hstate *h, struct folio *folio,
> 		       bool adjust_surplus)
> {
> +	bool cma = folio_test_hugetlb_cma(folio);
> 	int nid = folio_nid(folio);
> 
> 	VM_BUG_ON_FOLIO(!folio_test_hugetlb_vmemmap_optimized(folio), folio);
> @@ -1453,9 +1454,11 @@ void add_hugetlb_folio(struct hstate *h, struct folio *folio,
> 	}
> 
> 	__folio_set_hugetlb(folio);
> -	/* Clear all folio->private flags except hugetlb_vmemmap_optimized. */
> +	/* Preserve the vmemmap optimization state and CMA allocation origin. */
> 	folio->private = NULL;
> 	folio_set_hugetlb_vmemmap_optimized(folio);
> +	if (cma)
> +		folio_set_hugetlb_cma(folio);
> 
> 	arch_clear_hugetlb_flags(folio);
> 	enqueue_hugetlb_folio(h, folio);
> -- 
> 2.53.0
> 

Re: [PATCH 2/3] mm/hugetlb: preserve CMA flag when returning folios to the pool

[Zi Yan]

On 28 Oct 2026, at 9:56, Muchun Song wrote:

>> On Oct 28, 2026, at 19:33, Kaitao Cheng <kaitao.cheng@linux.dev> wrote:
>>
>> From: Kaitao Cheng <chengkaitao@kylinos.cn>
>>
>> When vmemmap restoration fails, add_hugetlb_folio() returns the folio
>> to the hugetlb pool. It clears folio->private and restores only
>> HPG_vmemmap_optimized, losing HPG_cma on CMA-allocated folios.
>>
>> A later successful free then bypasses hugetlb_cma_free_frozen_folio()
>> and returns the pages through free_frozen_pages(). The CMA allocation
>> bitmap remains set, so the range is unavailable for future CMA
>> allocations.
>>
>> Save the CMA flag before resetting folio->private and restore it along
>> with the vmemmap optimization flag.
>
> We already knew this issue. Cc Zi Yan. Seems he hasn't sent a fix.

Thanks. No, I have not formed a proper patch yet.

>
> But we want a different solution. See:
>
> https://lore.kernel.org/linux-mm/A1A14368-7A43-40ED-B2AB-C2EC2C5FFB1D@linux.dev/

Feel free to grab my version from the link and add
VM_WARN_ON_ONCE_FOLIO()s Muchun suggested. You can add
Suggested-by: Zi Yan <ziy@nvidia.com> in your patch.

Thanks.

>
> Thanks,
> Muchun
>
>>
>> Fixes: d2d786714080 ("mm/hugetlb: enable bootmem allocation from CMA areas")
>> Closes: https://sashiko.dev/#/bug/linux-5363eb09-9c3c-4f8b-9f3c-a998a8c4b258
>> Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
>> ---
>> mm/hugetlb.c | 5 ++++-
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>> index 4a008afaedf9..8da4bab676c9 100644
>> --- a/mm/hugetlb.c
>> +++ b/mm/hugetlb.c
>> @@ -1437,6 +1437,7 @@ void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
>> void add_hugetlb_folio(struct hstate *h, struct folio *folio,
>> 		       bool adjust_surplus)
>> {
>> +	bool cma = folio_test_hugetlb_cma(folio);
>> 	int nid = folio_nid(folio);
>>
>> 	VM_BUG_ON_FOLIO(!folio_test_hugetlb_vmemmap_optimized(folio), folio);
>> @@ -1453,9 +1454,11 @@ void add_hugetlb_folio(struct hstate *h, struct folio *folio,
>> 	}
>>
>> 	__folio_set_hugetlb(folio);
>> -	/* Clear all folio->private flags except hugetlb_vmemmap_optimized. */
>> +	/* Preserve the vmemmap optimization state and CMA allocation origin. */
>> 	folio->private = NULL;
>> 	folio_set_hugetlb_vmemmap_optimized(folio);
>> +	if (cma)
>> +		folio_set_hugetlb_cma(folio);
>>
>> 	arch_clear_hugetlb_flags(folio);
>> 	enqueue_hugetlb_folio(h, folio);
>> -- 
>> 2.53.0
>>


Best Regards,
Yan, Zi

[PATCH 3/3] mm/hugetlb: fix NR_HUGETLB accounting for unused surplus folios

[Kaitao Cheng]

From: Kaitao Cheng <chengkaitao@kylinos.cn>

free_huge_folio() unconditionally decrements NR_HUGETLB, but freshly
allocated surplus folios can be freed before the counter has been
incremented.

This happens when alloc_surplus_hugetlb_folio() rejects a fresh folio
after its second surplus limit check. It also happens when
gather_surplus_pages() frees folios because a reservation cannot be
satisfied, or because concurrent pool changes leave excess folios
after the reservation succeeds.

These paths leave unmatched decrements, causing HugeTLB usage to be
underreported and potentially underflowing the counter.

Increment NR_HUGETLB before calling free_huge_folio() in both paths
to balance its unconditional decrement.

Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
Closes: https://sashiko.dev/#/bug/linux-860a3b8f-0b07-4cd5-a122-aa2288463bee
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
---
 mm/hugetlb.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 8da4bab676c9..d564e6935e61 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2132,6 +2132,8 @@ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h,
 	if (h->surplus_huge_pages >= h->nr_overcommit_huge_pages) {
 		folio_set_hugetlb_temporary(folio);
 		spin_unlock_irq(&hugetlb_lock);
+		/* Balance the NR_HUGETLB decrement in free_huge_folio(). */
+		lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
 		free_huge_folio(folio);
 		return NULL;
 	}
@@ -2376,9 +2378,13 @@ static int gather_surplus_pages(struct hstate *h, long delta)
 	/*
 	 * Free unnecessary surplus pages to the buddy allocator.
 	 * Pages have no ref count, call free_huge_folio directly.
+	 * These pages were not accounted in NR_HUGETLB, so balance the
+	 * decrement in free_huge_folio().
 	 */
-	list_for_each_entry_safe(folio, tmp, &surplus_list, lru)
+	list_for_each_entry_safe(folio, tmp, &surplus_list, lru) {
+		lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h));
 		free_huge_folio(folio);
+	}
 	spin_lock_irq(&hugetlb_lock);
 
 	return ret;
-- 
2.53.0

Re: [PATCH 3/3] mm/hugetlb: fix NR_HUGETLB accounting for unused surplus folios

[Oscar Salvador (SUSE)]

On Wed, Oct 28, 2026 at 07:33:19PM +0800, Kaitao Cheng wrote:
> From: Kaitao Cheng <chengkaitao@kylinos.cn>
> 
> free_huge_folio() unconditionally decrements NR_HUGETLB, but freshly
> allocated surplus folios can be freed before the counter has been
> incremented.
> 
> This happens when alloc_surplus_hugetlb_folio() rejects a fresh folio
> after its second surplus limit check. It also happens when
> gather_surplus_pages() frees folios because a reservation cannot be
> satisfied, or because concurrent pool changes leave excess folios
> after the reservation succeeds.
> 
> These paths leave unmatched decrements, causing HugeTLB usage to be
> underreported and potentially underflowing the counter.
> 
> Increment NR_HUGETLB before calling free_huge_folio() in both paths
> to balance its unconditional decrement.
> 
> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
> Closes: https://sashiko.dev/#/bug/linux-860a3b8f-0b07-4cd5-a122-aa2288463bee
> Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>

I feel like the whole hugetlb accounting is a bit of a nightmare.
I wish we could teach free_huge_folio to ignore accounting-wise these pages
but not sure if that would be any prettier/easier.

I will have another look later on.

 

-- 
Oscar Salvador
SUSE Labs

Re: [PATCH 3/3] mm/hugetlb: fix NR_HUGETLB accounting for unused surplus folios

[Muchun Song]

> On Oct 28, 2026, at 23:01, Oscar Salvador (SUSE) <osalvador@kernel.org> wrote:
> 
> On Wed, Oct 28, 2026 at 07:33:19PM +0800, Kaitao Cheng wrote:
>> From: Kaitao Cheng <chengkaitao@kylinos.cn>
>> 
>> free_huge_folio() unconditionally decrements NR_HUGETLB, but freshly
>> allocated surplus folios can be freed before the counter has been
>> incremented.
>> 
>> This happens when alloc_surplus_hugetlb_folio() rejects a fresh folio
>> after its second surplus limit check. It also happens when
>> gather_surplus_pages() frees folios because a reservation cannot be
>> satisfied, or because concurrent pool changes leave excess folios
>> after the reservation succeeds.
>> 
>> These paths leave unmatched decrements, causing HugeTLB usage to be
>> underreported and potentially underflowing the counter.
>> 
>> Increment NR_HUGETLB before calling free_huge_folio() in both paths
>> to balance its unconditional decrement.
>> 
>> Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg")
>> Closes: https://sashiko.dev/#/bug/linux-860a3b8f-0b07-4cd5-a122-aa2288463bee
>> Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
> 
> I feel like the whole hugetlb accounting is a bit of a nightmare.

Can't agree more.

> I wish we could teach free_huge_folio to ignore accounting-wise these pages
> but not sure if that would be any prettier/easier.

My thinking is that we should not use lruvec_stat_mod_folio(). Instead, we
should use mod_node_page_state() and mod_memcg_lruvec_state()
independently. In that case, when a new hugetlb page is allocated, we could
account for it directly via mod_node_page_state(). The reason is that
lruvec_stat_mod_folio() also accounts per-memcg NR_HUGETLB; only by
splitting the two can we minimize the problems introduced by accounting. For
callers that need to consider memcg, mod_memcg_lruvec_state() can be used.

I expect that after some cleanup, there will be only one place that needs to
handle per-node NR_HUGETLB.

Thoughts?

Thanks,
Muchun


> I will have another look later on.
> 
> 
> 
> -- 
> Oscar Salvador
> SUSE Labs