All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qi Zheng <qi.zheng@linux.dev>
To: "David Hildenbrand (Arm)" <david@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	muchun.song@linux.dev, akpm@linux-foundation.org,
	osalvador@suse.de, azpijr@gmail.com, lance.yang@linux.dev,
	Qi Zheng <zhengqi.arch@bytedance.com>
Subject: Re: [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues
Date: Tue, 17 Nov 2026 14:55:36 +0800	[thread overview]
Message-ID: <c3906ff3-c5ee-4672-8bab-2318185268c8@linux.dev> (raw)
In-Reply-To: <1e0aa28d-f81b-411f-92a3-6038c3961443@kernel.org>

Hi David,

On 11/16/26 8:07 PM, David Hildenbrand (Arm) wrote:
> 
> Hi,
> 
> sorry somehow I missed your mail.
> 
> 
>>>
>>>>
>>>> Likely cma_reserve_early() must be taught to align (and maybe consume an order
>>>> instead of a size), similar to how the dynamic cma allocations work.
>>>
>>> However, simply adding alignment would leave gaps that the
>>> current activation code cannot distinguish from actual reservations.
>>>
>>> The early allocation path would become:
>>>
>>> hugetlb_cma_alloc_bootmem()
>>>      -> cma_reserve_early()
>>>           pfn = ALIGN(cmr->early_pfn, nr_pages);
>>>           cmr->early_pfn = pfn + nr_pages;
>>>           cma->available_count -= nr_pages;
>>>
>>> For a CMA range starting at 4 GiB, reserving 32 MiB followed by 1 GiB
>>> would produce:
>>>
>>> 4 GiB       4 GiB + 32 MiB             5 GiB             6 GiB
>>> | 32 MiB used |------ 992 MiB gap ------| 1 GiB used       |
>>>                                                            ^
>>>                                                        early_pfn
>>>
>>> Later, cma_activate_area() marks the entire prefix below early_pfn as
>>> allocated, including the gap:
>>>
>>> count = early_pfn[r] - cmr->base_pfn;
>>> bitmap_count = cma_bitmap_pages_to_bits(cma, count);
>>> bitmap_set(cmr->bitmap, 0, bitmap_count);
>>>
>>> It only initializes and frees pageblocks after that cursor:
>>>
>>> for (pfn = early_pfn[r]; pfn < cmr->base_pfn + cmr->count;
>>>         pfn += pageblock_nr_pages)
>>> 	init_cma_reserved_pageblock(pfn_to_page(pfn));
>>>
>>> The gap therefore remains reserved and marked allocated. Releasing the
>>> two HugeTLB folios only clears their own ranges through
>>> cma_release_frozen(); neither release covers the gap. Those 992 MiB
>>> would be permanently lost from usable CMA capacity.
>>>
>>> So we would also need to track the gaps, clear their bitmap bits during
>>> activation, and initialize them as free CMA pageblocks.
> 
> Ugh, what a hacky mess people created.
> 
>>>
>>>>
>>>
>>> Another possibility would be to give 32 MiB pages their own pool,
>>> similar to the separate PMD-sized pool proposed in the other series. On
>>> arm64 with 4 KiB base pages, 2 MiB, 32 MiB and 1 GiB allocations would
>>> then use separate pools.
> 
> That would be cleanest. Would it break existing setups?

Yes, this requires an existing user to explicitly configure a 32M memory
pool.

But in practice there shouldn't be such a user today, otherwise the
misalignment issue would have surfaced long ago, wouldn't it?

> 
> 
> Another quick-fix idea could be to always allocate larger folios first.
> 
> That way we should avoid any holes, right>

Ah, this looks feasible. How about the following?

(If that looks good, this could go as a stable-first fix.)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 6fba92b0f6749..32e65ce347c90 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -4556,6 +4556,22 @@ void __init hugetlb_bootmem_set_nodes(void)
  		node_set(nid, hugetlb_bootmem_nodes);
  }

+/* Return the largest early-CMA hstate below the given order. */
+static struct hstate * __init
+hugetlb_next_early_cma_hstate(unsigned int order)
+{
+	struct hstate *h, *next = NULL;
+
+	for_each_hstate(h) {
+		if (!hugetlb_early_cma(h) || huge_page_order(h) >= order)
+			continue;
+		if (!next || huge_page_order(h) > huge_page_order(next))
+			next = h;
+	}
+
+	return next;
+}
+
  void __init hugetlb_bootmem_alloc(void)
  {
  	struct hstate *h;
@@ -4571,9 +4587,19 @@ void __init hugetlb_bootmem_alloc(void)
  	for_each_hstate(h) {
  		h->next_nid_to_alloc = first_online_node;

-		if (hstate_is_gigantic(h))
+		if (hstate_is_gigantic(h) && !hugetlb_early_cma(h))
  			hugetlb_hstate_alloc_pages(h);
  	}
+
+	/*
+	 * CMA ranges are aligned to the largest supported size. Allocate larger
+	 * sizes first to keep the early reservation cursor aligned without gaps.
+	 */
+	h = hugetlb_next_early_cma_hstate(UINT_MAX);
+	while (h) {
+		hugetlb_hstate_alloc_pages(h);
+		h = hugetlb_next_early_cma_hstate(huge_page_order(h));
+	}
  }

  /*

> 
>>>
>>> This could work during early boot too: hugetlb_cma_reserve() establishes
>>> the CMA areas and their ranges before hugetlb_bootmem_alloc() runs.
>>> Although the bitmaps are allocated later, each range already has its
>>> own early_pfn. With naturally aligned ranges and one allocation size
>>> per pool, the cursors would stay aligned without creating gaps.
>>>
>>> What do you think?
>>
>> Hi David, gentle ping
>>
>> I’d like to update to v5 so that I can continue advancing the support
> 
> Feel free to fix this issue here independently from the current series.

OK. I'll go ahead and post the v5.

> 
>> for 2M hugetlb CMA. Regarding this issue, would you like me to include a
>> fix in v5? If you agree with the "split pool" approach, we can handle it
>> as part of the patch set for 2M hugetlb CMA support.
> 
> 
> I think we want a stable-first fix for that.

Got it.

Thanks,
Qi

> 
> 



      reply	other threads:[~2026-11-17  6:55 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-07 10:30 [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues Qi Zheng
2026-11-07 10:30 ` [PATCH v3 1/3] x86: hugetlb: disable default CMA reservations on 32-bit kernel Qi Zheng
2026-11-07 10:30 ` [PATCH v3 2/3] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios Qi Zheng
2026-11-07 17:58   ` David Hildenbrand (Arm)
2026-11-09  3:10     ` Qi Zheng
2026-11-11  7:43       ` David Hildenbrand (Arm)
2026-11-11 10:26         ` Qi Zheng
2026-11-11 22:26           ` Frank van der Linden
2026-11-12  3:27             ` Qi Zheng
2026-11-12  8:36             ` David Hildenbrand (Arm)
2026-11-12  8:40           ` David Hildenbrand (Arm)
2026-11-07 10:30 ` [PATCH v3 3/3] mm: hugetlb: fix GFP mask propagation to CMA Qi Zheng
2026-11-07 17:02   ` Oscar Salvador (SUSE)
2026-11-09  3:10     ` Qi Zheng
2026-11-11 22:05 ` [PATCH v3 0/3] mm: hugetlb: fix CMA-related issues David Hildenbrand (Arm)
2026-11-12  3:25   ` Qi Zheng
2026-11-16  2:01     ` Qi Zheng
2026-11-16 12:07       ` David Hildenbrand (Arm)
2026-11-17  6:55         ` Qi Zheng [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c3906ff3-c5ee-4672-8bab-2318185268c8@linux.dev \
    --to=qi.zheng@linux.dev \
    --cc=akpm@linux-foundation.org \
    --cc=azpijr@gmail.com \
    --cc=david@kernel.org \
    --cc=lance.yang@linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=muchun.song@linux.dev \
    --cc=osalvador@suse.de \
    --cc=zhengqi.arch@bytedance.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.