Linux-mm Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Qi Zheng <qi.zheng@linux.dev>
To: "David Hildenbrand (Arm)" <david@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	muchun.song@linux.dev, akpm@linux-foundation.org,
	osalvador@suse.de, azpijr@gmail.com, lance.yang@linux.dev,
	Qi Zheng <zhengqi.arch@bytedance.com>
Subject: Re: [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues
Date: Tue, 17 Nov 2026 14:55:36 +0800	[thread overview]
Message-ID: <c3906ff3-c5ee-4672-8bab-2318185268c8@linux.dev> (raw)
In-Reply-To: <1e0aa28d-f81b-411f-92a3-6038c3961443@kernel.org>

Hi David,

On 11/16/26 8:07 PM, David Hildenbrand (Arm) wrote:
> 
> Hi,
> 
> sorry somehow I missed your mail.
> 
> 
>>>
>>>>
>>>> Likely cma_reserve_early() must be taught to align (and maybe consume an order
>>>> instead of a size), similar to how the dynamic cma allocations work.
>>>
>>> However, simply adding alignment would leave gaps that the
>>> current activation code cannot distinguish from actual reservations.
>>>
>>> The early allocation path would become:
>>>
>>> hugetlb_cma_alloc_bootmem()
>>>      -> cma_reserve_early()
>>>           pfn = ALIGN(cmr->early_pfn, nr_pages);
>>>           cmr->early_pfn = pfn + nr_pages;
>>>           cma->available_count -= nr_pages;
>>>
>>> For a CMA range starting at 4 GiB, reserving 32 MiB followed by 1 GiB
>>> would produce:
>>>
>>> 4 GiB       4 GiB + 32 MiB             5 GiB             6 GiB
>>> | 32 MiB used |------ 992 MiB gap ------| 1 GiB used       |
>>>                                                            ^
>>>                                                        early_pfn
>>>
>>> Later, cma_activate_area() marks the entire prefix below early_pfn as
>>> allocated, including the gap:
>>>
>>> count = early_pfn[r] - cmr->base_pfn;
>>> bitmap_count = cma_bitmap_pages_to_bits(cma, count);
>>> bitmap_set(cmr->bitmap, 0, bitmap_count);
>>>
>>> It only initializes and frees pageblocks after that cursor:
>>>
>>> for (pfn = early_pfn[r]; pfn < cmr->base_pfn + cmr->count;
>>>         pfn += pageblock_nr_pages)
>>> 	init_cma_reserved_pageblock(pfn_to_page(pfn));
>>>
>>> The gap therefore remains reserved and marked allocated. Releasing the
>>> two HugeTLB folios only clears their own ranges through
>>> cma_release_frozen(); neither release covers the gap. Those 992 MiB
>>> would be permanently lost from usable CMA capacity.
>>>
>>> So we would also need to track the gaps, clear their bitmap bits during
>>> activation, and initialize them as free CMA pageblocks.
> 
> Ugh, what a hacky mess people created.
> 
>>>
>>>>
>>>
>>> Another possibility would be to give 32 MiB pages their own pool,
>>> similar to the separate PMD-sized pool proposed in the other series. On
>>> arm64 with 4 KiB base pages, 2 MiB, 32 MiB and 1 GiB allocations would
>>> then use separate pools.
> 
> That would be cleanest. Would it break existing setups?

Yes, this requires an existing user to explicitly configure a 32M memory
pool.

But in practice there shouldn't be such a user today, otherwise the
misalignment issue would have surfaced long ago, wouldn't it?

> 
> 
> Another quick-fix idea could be to always allocate larger folios first.
> 
> That way we should avoid any holes, right>

Ah, this looks feasible. How about the following?

(If that looks good, this could go as a stable-first fix.)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 6fba92b0f6749..32e65ce347c90 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -4556,6 +4556,22 @@ void __init hugetlb_bootmem_set_nodes(void)
  		node_set(nid, hugetlb_bootmem_nodes);
  }

+/* Return the largest early-CMA hstate below the given order. */
+static struct hstate * __init
+hugetlb_next_early_cma_hstate(unsigned int order)
+{
+	struct hstate *h, *next = NULL;
+
+	for_each_hstate(h) {
+		if (!hugetlb_early_cma(h) || huge_page_order(h) >= order)
+			continue;
+		if (!next || huge_page_order(h) > huge_page_order(next))
+			next = h;
+	}
+
+	return next;
+}
+
  void __init hugetlb_bootmem_alloc(void)
  {
  	struct hstate *h;
@@ -4571,9 +4587,19 @@ void __init hugetlb_bootmem_alloc(void)
  	for_each_hstate(h) {
  		h->next_nid_to_alloc = first_online_node;

-		if (hstate_is_gigantic(h))
+		if (hstate_is_gigantic(h) && !hugetlb_early_cma(h))
  			hugetlb_hstate_alloc_pages(h);
  	}
+
+	/*
+	 * CMA ranges are aligned to the largest supported size. Allocate larger
+	 * sizes first to keep the early reservation cursor aligned without gaps.
+	 */
+	h = hugetlb_next_early_cma_hstate(UINT_MAX);
+	while (h) {
+		hugetlb_hstate_alloc_pages(h);
+		h = hugetlb_next_early_cma_hstate(huge_page_order(h));
+	}
  }

  /*

> 
>>>
>>> This could work during early boot too: hugetlb_cma_reserve() establishes
>>> the CMA areas and their ranges before hugetlb_bootmem_alloc() runs.
>>> Although the bitmaps are allocated later, each range already has its
>>> own early_pfn. With naturally aligned ranges and one allocation size
>>> per pool, the cursors would stay aligned without creating gaps.
>>>
>>> What do you think?
>>
>> Hi David, gentle ping
>>
>> I’d like to update to v5 so that I can continue advancing the support
> 
> Feel free to fix this issue here independently from the current series.

OK. I'll go ahead and post the v5.

> 
>> for 2M hugetlb CMA. Regarding this issue, would you like me to include a
>> fix in v5? If you agree with the "split pool" approach, we can handle it
>> as part of the patch set for 2M hugetlb CMA support.
> 
> 
> I think we want a stable-first fix for that.

Got it.

Thanks,
Qi

> 
> 



      reply	other threads:[~2026-11-17  6:55 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-07 10:30 [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues Qi Zheng
2026-11-07 10:30 ` [PATCH v3 1/3] x86: hugetlb: disable default CMA reservations on 32-bit kernel Qi Zheng
2026-11-07 10:30 ` [PATCH v3 2/3] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios Qi Zheng
2026-11-07 17:58   ` David Hildenbrand (Arm)
2026-11-09  3:10     ` Qi Zheng
2026-11-11  7:43       ` David Hildenbrand (Arm)
2026-11-11 10:26         ` Qi Zheng
2026-11-11 22:26           ` Frank van der Linden
2026-11-12  3:27             ` Qi Zheng
2026-11-12  8:36             ` David Hildenbrand (Arm)
2026-11-12  8:40           ` David Hildenbrand (Arm)
2026-11-07 10:30 ` [PATCH v3 3/3] mm: hugetlb: fix GFP mask propagation to CMA Qi Zheng
2026-11-07 17:02   ` Oscar Salvador (SUSE)
2026-11-09  3:10     ` Qi Zheng
2026-11-11 22:05 ` [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues David Hildenbrand (Arm)
2026-11-12  3:25   ` Qi Zheng
2026-11-16  2:01     ` Qi Zheng
2026-11-16 12:07       ` David Hildenbrand (Arm)
2026-11-17  6:55         ` Qi Zheng [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c3906ff3-c5ee-4672-8bab-2318185268c8@linux.dev \
    --to=qi.zheng@linux.dev \
    --cc=akpm@linux-foundation.org \
    --cc=azpijr@gmail.com \
    --cc=david@kernel.org \
    --cc=lance.yang@linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=muchun.song@linux.dev \
    --cc=osalvador@suse.de \
    --cc=zhengqi.arch@bytedance.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox