[PATCH v2 0/4] mm: hugetlb: fix CMA-related issues

[PATCH v2 0/4] mm: hugetlb: fix CMA-related issues

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Hi all,

This series fixes four CMA-related issues in HugeTLB reservation,
long-term pinning and allocation:

 - Avoid ineffective CMA reservations on 32-bit x86 kernels running on
   CPUs that advertise 1 GiB page support.
 - Allow long-term pins of explicitly CMA-allocated HugeTLB folios,
   which already own their CMA bitmap ranges.
 - Preserve the caller's GFP mask when allocating from CMA.
 - Enforce cpuset restrictions on gigantic surplus allocations while
   preserving memory policy semantics.

Changes in v2:
 - Replace the PF_MEMALLOC_PIN allocation restriction with a GUP exception
   for HugeTLB folios marked HPG_cma. (suggested by Muchun Song)
 - Pass the caller's GFP mask through cma_alloc_frozen_compound() instead
   of skipping CMA when direct reclaim is disallowed.
   (suggested by sashiko)
 - Remove unnecessary revalidation of successfully allocated folios after
   cpuset changes. Retry only failed allocations.
   (pointed by Jose A. Perez de Azpillaga)
 - collect Reviewed-by and Acked-by.
 - Rebase onto next-20261104.

Qi Zheng (4):
  x86: hugetlb: disable default CMA reservations on 32-bit kernel
  mm: gup: allow long-term pins of CMA-allocated HugeTLB folios
  mm: hugetlb: fix GFP mask propagation to CMA
  mm: hugetlb: fix cpuset violations in gigantic surplus allocations

 arch/x86/mm/hugetlbpage.c |  2 +-
 include/linux/cma.h       |  2 +-
 include/linux/mm.h        | 29 +----------------------------
 mm/cma.c                  |  4 ++--
 mm/gup.c                  | 31 +++++++++++++++++++++++++++++++
 mm/hugetlb.c              | 25 ++++++++++++++++++-------
 mm/hugetlb_cma.c          |  4 ++--
 7 files changed, 56 insertions(+), 41 deletions(-)


base-commit: 8b7179210b22f8e48a82c5ceacb9d56a9c4cfec4
-- 
2.54.0

[PATCH v2 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.
arch_hugetlb_cma_order() checks only this feature and returns a nonzero
order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
serve the intended gigantic HugeTLB allocations.

On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
warning that hugetlb_cma is attempting to reserve a non-gigantic order.

Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
order to avoid these ineffective reservations.

Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
Cc: stable@vger.kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
Acked-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
---
 arch/x86/mm/hugetlbpage.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
index 3b26621c91286..70726a99ede7b 100644
--- a/arch/x86/mm/hugetlbpage.c
+++ b/arch/x86/mm/hugetlbpage.c
@@ -45,7 +45,7 @@ arch_initcall(gigantic_pages_init);
 
 unsigned int __init arch_hugetlb_cma_order(void)
 {
-	if (boot_cpu_has(X86_FEATURE_GBPAGES))
+	if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_GBPAGES))
 		return PUD_SHIFT - PAGE_SHIFT;
 
 	return 0;
-- 
2.54.0

Re: [PATCH v2 1/4] x86: hugetlb: disable default CMA reservations on 32-bit kernel

[Oscar Salvador (SUSE)]

On Thu, Nov 05, 2026 at 12:07:43PM +0800, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES.
> arch_hugetlb_cma_order() checks only this feature and returns a nonzero
> order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64.
> Specifying hugetlb_cma= can therefore reserve a CMA area that cannot
> serve the intended gigantic HugeTLB allocations.
> 
> On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is
> equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the
> VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(),
> warning that hugetlb_cma is attempting to reserve a non-gigantic order.
> 
> Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA
> order to avoid these ineffective reservations.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
> Acked-by: Muchun Song <muchun.song@linux.dev>
> Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

Acked-by: Oscar Salvador <osalvador@suse.de>

> ---
>  arch/x86/mm/hugetlbpage.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c
> index 3b26621c91286..70726a99ede7b 100644
> --- a/arch/x86/mm/hugetlbpage.c
> +++ b/arch/x86/mm/hugetlbpage.c
> @@ -45,7 +45,7 @@ arch_initcall(gigantic_pages_init);
>  
>  unsigned int __init arch_hugetlb_cma_order(void)
>  {
> -	if (boot_cpu_has(X86_FEATURE_GBPAGES))
> +	if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_GBPAGES))
>  		return PUD_SHIFT - PAGE_SHIFT;
>  
>  	return 0;
> -- 
> 2.54.0
> 

-- 
Oscar Salvador
SUSE Labs

[PATCH v2 2/4] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

Long-term pinning treats explicitly CMA-allocated HugeTLB folios as
unpinnable, causing unnecessary migration or pinning failures.

These folios already own their CMA bitmap ranges, and memory blocks
overlapping CMA areas cannot be offlined. Allow HugeTLB folios with
HPG_cma to be pinned directly, including in ZONE_MOVABLE.

Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
Cc: stable@vger.kernel.org
Suggested-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 include/linux/mm.h | 29 +----------------------------
 mm/gup.c           | 31 +++++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 28 deletions(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 2d0b8a4c9974f..75dd40265d3f4 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -2975,35 +2975,8 @@ static inline bool is_zero_folio(const struct folio *folio)
 	return is_zero_page(&folio->page);
 }
 
-/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
 #ifdef CONFIG_MIGRATION
-static inline bool folio_is_longterm_pinnable(struct folio *folio)
-{
-#ifdef CONFIG_CMA
-	int mt = folio_migratetype(folio);
-
-	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
-		return false;
-#endif
-	/* The zero page can be "pinned" but gets special handling. */
-	if (is_zero_folio(folio))
-		return true;
-
-	/* Coherent device memory must always allow eviction. */
-	if (folio_is_device_coherent(folio))
-		return false;
-
-	/*
-	 * Filesystems can only tolerate transient delays to truncate and
-	 * hole-punch operations
-	 */
-	if (folio_is_fsdax(folio))
-		return false;
-
-	/* Otherwise, non-movable zone folios can be pinned. */
-	return !folio_is_zone_movable(folio);
-
-}
+bool folio_is_longterm_pinnable(struct folio *folio);
 #else
 static inline bool folio_is_longterm_pinnable(struct folio *folio)
 {
diff --git a/mm/gup.c b/mm/gup.c
index 2b405cce2ca5c..e8503141bb275 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2216,6 +2216,37 @@ struct page *get_dump_page(unsigned long addr, int *locked)
 
 #ifdef CONFIG_MIGRATION
 
+bool folio_is_longterm_pinnable(struct folio *folio)
+{
+#ifdef CONFIG_CMA
+	int mt = folio_migratetype(folio);
+
+	/* Explicit HugeTLB CMA allocations own their bitmap ranges. */
+	if (folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio))
+		return true;
+
+	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
+		return false;
+#endif
+	/* The zero page can be "pinned" but gets special handling. */
+	if (is_zero_folio(folio))
+		return true;
+
+	/* Coherent device memory must always allow eviction. */
+	if (folio_is_device_coherent(folio))
+		return false;
+
+	/*
+	 * Filesystems can only tolerate transient delays to truncate and
+	 * hole-punch operations
+	 */
+	if (folio_is_fsdax(folio))
+		return false;
+
+	/* Otherwise, non-movable zone folios can be pinned. */
+	return !folio_is_zone_movable(folio);
+}
+
 /*
  * An array of either pages or folios ("pofs"). Although it may seem tempting to
  * avoid this complication, by simply interpreting a list of folios as a list of
-- 
2.54.0

Re: [PATCH v2 2/4] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios

[Muchun Song]

> On Nov 5, 2026, at 12:07, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> Long-term pinning treats explicitly CMA-allocated HugeTLB folios as
> unpinnable, causing unnecessary migration or pinning failures.
> 
> These folios already own their CMA bitmap ranges, and memory blocks
> overlapping CMA areas cannot be offlined. Allow HugeTLB folios with
> HPG_cma to be pinned directly, including in ZONE_MOVABLE.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Suggested-by: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

Acked-by: Muchun Song <muchun.song@linux.dev>

But a nit bellow.

> ---
> include/linux/mm.h | 29 +----------------------------
> mm/gup.c           | 31 +++++++++++++++++++++++++++++++
> 2 files changed, 32 insertions(+), 28 deletions(-)
> 
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 2d0b8a4c9974f..75dd40265d3f4 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -2975,35 +2975,8 @@ static inline bool is_zero_folio(const struct folio *folio)
> 	return is_zero_page(&folio->page);
> }
> 
> -/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
> #ifdef CONFIG_MIGRATION
> -static inline bool folio_is_longterm_pinnable(struct folio *folio)
> -{
> -#ifdef CONFIG_CMA
> -	int mt = folio_migratetype(folio);
> -
> -	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
> -		return false;
> -#endif
> -	/* The zero page can be "pinned" but gets special handling. */
> -	if (is_zero_folio(folio))
> -		return true;
> -
> -	/* Coherent device memory must always allow eviction. */
> -	if (folio_is_device_coherent(folio))
> -		return false;
> -
> -	/*
> -	 * Filesystems can only tolerate transient delays to truncate and
> -	 * hole-punch operations
> -	 */
> -	if (folio_is_fsdax(folio))
> -		return false;
> -
> -	/* Otherwise, non-movable zone folios can be pinned. */
> -	return !folio_is_zone_movable(folio);
> -
> -}
> +bool folio_is_longterm_pinnable(struct folio *folio);
> #else
> static inline bool folio_is_longterm_pinnable(struct folio *folio)
> {
> diff --git a/mm/gup.c b/mm/gup.c
> index 2b405cce2ca5c..e8503141bb275 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -2216,6 +2216,37 @@ struct page *get_dump_page(unsigned long addr, int *locked)
> 
> #ifdef CONFIG_MIGRATION
> 
> +bool folio_is_longterm_pinnable(struct folio *folio)
> +{
> +#ifdef CONFIG_CMA
> +	int mt = folio_migratetype(folio);
> +
> +	/* Explicit HugeTLB CMA allocations own their bitmap ranges. */

The description of "bitmap ranges" is weird, making me think what's the meaning of
bitmap. Please use more generic description here.

> +	if (folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio))
> +		return true;
> +
> +	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
> +		return false;
> +#endif
> +	/* The zero page can be "pinned" but gets special handling. */
> +	if (is_zero_folio(folio))
> +		return true;
> +
> +	/* Coherent device memory must always allow eviction. */
> +	if (folio_is_device_coherent(folio))
> +		return false;
> +
> +	/*
> +	 * Filesystems can only tolerate transient delays to truncate and
> +	 * hole-punch operations
> +	 */
> +	if (folio_is_fsdax(folio))
> +		return false;
> +
> +	/* Otherwise, non-movable zone folios can be pinned. */
> +	return !folio_is_zone_movable(folio);
> +}
> +
> /*
>  * An array of either pages or folios ("pofs"). Although it may seem tempting to
>  * avoid this complication, by simply interpreting a list of folios as a list of
> -- 
> 2.54.0
> 

Re: [PATCH v2 2/4] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios

[Oscar Salvador (SUSE)]

On Thu, Nov 05, 2026 at 12:07:44PM +0800, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> Long-term pinning treats explicitly CMA-allocated HugeTLB folios as
> unpinnable, causing unnecessary migration or pinning failures.
> 
> These folios already own their CMA bitmap ranges, and memory blocks
> overlapping CMA areas cannot be offlined. Allow HugeTLB folios with
> HPG_cma to be pinned directly, including in ZONE_MOVABLE.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Suggested-by: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

With the rewording suggested by Munchun:

Reviewed-by: Oscar Salvador <osalvador@suse.de>

 

-- 
Oscar Salvador
SUSE Labs

Re: [PATCH v2 2/4] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios

[Jose A. Perez de Azpillaga]

On 11/5/26 5:07 AM, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> Long-term pinning treats explicitly CMA-allocated HugeTLB folios as
> unpinnable, causing unnecessary migration or pinning failures.
> 
> These folios already own their CMA bitmap ranges, and memory blocks
> overlapping CMA areas cannot be offlined. Allow HugeTLB folios with
> HPG_cma to be pinned directly, including in ZONE_MOVABLE.
> 
> Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
> Cc: stable@vger.kernel.org
> Suggested-by: Muchun Song <muchun.song@linux.dev>
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>


LGTM:

Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

-- 
cheers,
jose a. p-a

[PATCH v2 3/4] mm: hugetlb: fix GFP mask propagation to CMA

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

cma_alloc_frozen_compound() uses GFP_KERNEL, allowing direct reclaim
during the first MPOL_PREFERRED_MANY allocation attempt even though
the caller clears __GFP_DIRECT_RECLAIM.

Pass the caller's GFP mask through the helper so CMA respects its
reclaim constraints when trying the preferred nodes.

Fixes: 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
Cc: stable@vger.kernel.org
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 include/linux/cma.h | 2 +-
 mm/cma.c            | 4 ++--
 mm/hugetlb_cma.c    | 4 ++--
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/linux/cma.h b/include/linux/cma.h
index 8555d38a97b1a..4539ea5f2c4e8 100644
--- a/include/linux/cma.h
+++ b/include/linux/cma.h
@@ -53,7 +53,7 @@ extern bool cma_release(struct cma *cma, const struct page *pages, unsigned long
 
 struct page *cma_alloc_frozen(struct cma *cma, unsigned long count,
 		unsigned int align, bool no_warn);
-struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order);
+struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, gfp_t gfp);
 bool cma_release_frozen(struct cma *cma, const struct page *pages,
 		unsigned long count);
 
diff --git a/mm/cma.c b/mm/cma.c
index 9ec1d1fd5dff5..876635a56180e 100644
--- a/mm/cma.c
+++ b/mm/cma.c
@@ -933,9 +933,9 @@ struct page *cma_alloc_frozen(struct cma *cma, unsigned long count,
 	return __cma_alloc_frozen(cma, count, align, gfp);
 }
 
-struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order)
+struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, gfp_t gfp)
 {
-	gfp_t gfp = GFP_KERNEL | __GFP_COMP | __GFP_NOWARN;
+	gfp |= __GFP_COMP | __GFP_NOWARN;
 
 	return __cma_alloc_frozen(cma, 1 << order, order, gfp);
 }
diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c
index 80b2407e9f077..307e5736ecdbe 100644
--- a/mm/hugetlb_cma.c
+++ b/mm/hugetlb_cma.c
@@ -69,14 +69,14 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
 	}
 
 	if (hugetlb_cma[nid] && node_isset(nid, *nmask))
-		page = cma_alloc_frozen_compound(hugetlb_cma[nid], order);
+		page = cma_alloc_frozen_compound(hugetlb_cma[nid], order, gfp_mask);
 
 	if (!page && !(gfp_mask & __GFP_THISNODE)) {
 		for_each_node_mask(node, *nmask) {
 			if (node == nid || !hugetlb_cma[node])
 				continue;
 
-			page = cma_alloc_frozen_compound(hugetlb_cma[node], order);
+			page = cma_alloc_frozen_compound(hugetlb_cma[node], order, gfp_mask);
 			if (page)
 				break;
 		}
-- 
2.54.0

Re: [PATCH v2 3/4] mm: hugetlb: fix GFP mask propagation to CMA

[Muchun Song]

> On Nov 5, 2026, at 12:07, Qi Zheng <qi.zheng@linux.dev> wrote:
> 
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> cma_alloc_frozen_compound() uses GFP_KERNEL, allowing direct reclaim
> during the first MPOL_PREFERRED_MANY allocation attempt even though
> the caller clears __GFP_DIRECT_RECLAIM.
> 
> Pass the caller's GFP mask through the helper so CMA respects its
> reclaim constraints when trying the preferred nodes.

More natural.

> 
> Fixes: 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

Acked-by: Muchun Song <muchun.song@linux.dev>

Thanks.

Re: [PATCH v2 3/4] mm: hugetlb: fix GFP mask propagation to CMA

[Oscar Salvador (SUSE)]

On Thu, Nov 05, 2026 at 12:07:45PM +0800, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> cma_alloc_frozen_compound() uses GFP_KERNEL, allowing direct reclaim
> during the first MPOL_PREFERRED_MANY allocation attempt even though
> the caller clears __GFP_DIRECT_RECLAIM.
> 
> Pass the caller's GFP mask through the helper so CMA respects its
> reclaim constraints when trying the preferred nodes.
> 
> Fixes: 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
...
> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -933,9 +933,9 @@ struct page *cma_alloc_frozen(struct cma *cma, unsigned long count,
>  	return __cma_alloc_frozen(cma, count, align, gfp);
>  }
>  
> -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order)
> +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, gfp_t gfp)
>  {
> -	gfp_t gfp = GFP_KERNEL | __GFP_COMP | __GFP_NOWARN;
> +	gfp |= __GFP_COMP | __GFP_NOWARN;

Do we really need __GFP_COMP | __GFP_NOWARN here?
It is my understanding that all allocations coming from hugetlb are
handled with htlb_alloc_mask(), and that should already provide
__GFP_COMP | __GFP_NOWARN.
We might have needed it when cma_alloc_frozen_compound() constructed its
own mask, but not it gets it from the caller, and that should have been
done via htlb_alloc_mask() up the chain.

So, any reason to have it here now?

 

-- 
Oscar Salvador
SUSE Labs

Re: [PATCH v2 3/4] mm: hugetlb: fix GFP mask propagation to CMA

[Jose A. Perez de Azpillaga]

On 11/5/26 5:07 AM, Qi Zheng wrote:
> From: Qi Zheng <zhengqi.arch@bytedance.com>
> 
> cma_alloc_frozen_compound() uses GFP_KERNEL, allowing direct reclaim
> during the first MPOL_PREFERRED_MANY allocation attempt even though
> the caller clears __GFP_DIRECT_RECLAIM.
> 
> Pass the caller's GFP mask through the helper so CMA respects its
> reclaim constraints when trying the preferred nodes.
> 
> Fixes: 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
> Cc: stable@vger.kernel.org
> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>

LGTM:

Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>

-- 
cheers,
jose a. p-a

[PATCH v2 4/4] mm: hugetlb: fix cpuset violations in gigantic surplus allocations

[Qi Zheng]

From: Qi Zheng <zhengqi.arch@bytedance.com>

CMA and the contiguous allocator trust a supplied nodemask without
checking the current cpuset. Gigantic surplus allocations can therefore
use nodes forbidden to the faulting thread.

For example, moving a non-leader thread to another cgroup v1 cpuset
updates its mems_allowed without rebinding the shared VMA policy:

  CPU0 (cpuset attach)             CPU1 (thread T)
  -------------------             ---------------
  T->mems_allowed = {0}
  /* Skip mpol_rebind_mm()
     for a non-leader. */
                                  /* VMA MPOL_BIND remains {0,1}. */
                                  hugetlb_cma_alloc_frozen_folio()
                                    /* node0 fails; node1 succeeds. */

Intersect the allocation mask with the current cpuset, preserving
MPOL_BIND and MPOL_PREFERRED_MANY fallback rules. Retry only failed
allocations when the cpuset changes.

Fixes: eb02f14c4a2b ("mm/hugetlb: allow overcommitting gigantic hugepages")
Cc: stable@vger.kernel.org
Reported-by: Lance Yang <lance.yang@linux.dev>
Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
---
 mm/hugetlb.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 6fba92b0f6749..09b64f7c11dd2 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -2181,22 +2181,33 @@ struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
 		gfp_t gfp_mask, struct mempolicy_interpreted *mpoli)
 {
 	struct folio *folio = NULL;
-	nodemask_t *nodemask = mpoli->nodemask;
+	nodemask_t alloc_nodemask;
+	unsigned int cpuset_mems_cookie;
+
+retry:
+	cpuset_mems_cookie = read_mems_allowed_begin();
+	alloc_nodemask = cpuset_current_mems_allowed;
+	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
+	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
+		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
 
 	if (mpoli->mode == MPOL_PREFERRED_MANY) {
 		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
 
+		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
 		folio = alloc_surplus_hugetlb_folio(h, gfp, mpoli->nid,
-						    nodemask);
+						    &alloc_nodemask);
 
-		/* Fallback to all nodes if page==NULL */
-		nodemask = NULL;
+		/* Fallback to all nodes allowed by the current cpuset */
+		alloc_nodemask = cpuset_current_mems_allowed;
 	}
 
-	if (!folio) {
+	if (!folio)
 		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, mpoli->nid,
-						    nodemask);
-	}
+						    &alloc_nodemask);
+
+	if (!folio && unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
+		goto retry;
 
 	return folio;
 }
-- 
2.54.0

Re: [PATCH v2 4/4] mm: hugetlb: fix cpuset violations in gigantic surplus allocations

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] mm: hugetlb: empty nodemask passed in alloc_buddy_hugetlb_folio()
- [Medium] mm: hugetlb: MPOL_BIND violation in alloc_buddy_hugetlb_folio() on cpuset race
--

commit 6e27dfcfc4f6238bea6b6e8639c15188d8d94295
Author: Qi Zheng <qi.zheng@linux.dev>

mm: hugetlb: fix cpuset violations in gigantic surplus allocations

This commit intersects the allocation nodemask in alloc_buddy_hugetlb_folio()
with cpuset_current_mems_allowed so gigantic surplus hugepage allocations do
not allocate from nodes outside the current task's cpuset. It also retries
failed allocations if cpuset_current_mems_allowed changes during the attempt.

> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 6fba92b0f6749..09b64f7c11dd2 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -2181,22 +2181,33 @@ struct folio *alloc_buddy_hugetlb_folio(struct hstate *h,
>  		gfp_t gfp_mask, struct mempolicy_interpreted *mpoli)
>  {
>  	struct folio *folio = NULL;
> -	nodemask_t *nodemask = mpoli->nodemask;
> +	nodemask_t alloc_nodemask;
> +	unsigned int cpuset_mems_cookie;
> +
> +retry:
> +	cpuset_mems_cookie = read_mems_allowed_begin();
> +	alloc_nodemask = cpuset_current_mems_allowed;
> +	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
> +	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
> +		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
>  
>  	if (mpoli->mode == MPOL_PREFERRED_MANY) {
>  		gfp_t gfp = gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL);
>  
> +		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);
>  		folio = alloc_surplus_hugetlb_folio(h, gfp, mpoli->nid,
> -						    nodemask);
> +						    &alloc_nodemask);

[Severity: Medium]
What happens here when an MPOL_PREFERRED_MANY policy's nodemask is disjoint
from cpuset_current_mems_allowed (for example, after moving a task or
non-leader thread to a cpuset on different NUMA nodes, since
mpol_rebind_preferred() does not remap pol->nodes)?

Unlike the MPOL_BIND branch above, this path does not check whether the
intersection is non-empty before calling alloc_surplus_hugetlb_folio() with
&alloc_nodemask. When alloc_nodemask is empty:

1. For gigantic hugepages, alloc_surplus_hugetlb_folio() performs a wasted
   pass through CMA and alloc_contig_frozen_pages() with an empty nodemask.

2. For non-gigantic hugepages, alloc_surplus_hugetlb_folio() ->
   alloc_fresh_hugetlb_folio() -> only_alloc_fresh_hugetlb_folio() ->
   alloc_buddy_frozen_folio() -> __alloc_frozen_pages_noprof() fails the fast
   path and enters __alloc_pages_slowpath(). In check_retry_cpuset():

   mm/page_alloc.c:check_retry_cpuset() {
   	if (cpusets_enabled() && ac->nodemask &&
   			!cpuset_nodemask_valid_mems_allowed(ac->nodemask)) {
   		ac->nodemask = NULL;
   		return true;
   	}
   }

   Because cpuset_nodemask_valid_mems_allowed() returns false for an empty
   nodemask, check_retry_cpuset() treats it as a raced MPOL_BIND nodemask,
   resets ac->nodemask to NULL, and restarts __alloc_pages_slowpath() with
   __GFP_DIRECT_RECLAIM masked off. That wakes kswapd across all nodes via
   wake_all_kswapds() with ac->nodemask == NULL, attempts an allocation on
   non-preferred fallback cpuset nodes at ALLOC_WMARK_MIN before the normal
   fallback pass at ALLOC_WMARK_LOW with reclaim/compaction runs, and
   increments HTLB_BUDDY_PGALLOC_FAIL in alloc_buddy_frozen_folio() if that
   first pass fails.

Should the first-pass alloc_surplus_hugetlb_folio() call be guarded by
if (nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask)) so that an
empty intersection skips straight to the cpuset fallback pass?

>  
> -		/* Fallback to all nodes if page==NULL */
> -		nodemask = NULL;
> +		/* Fallback to all nodes allowed by the current cpuset */
> +		alloc_nodemask = cpuset_current_mems_allowed;
>  	}
>  
> -	if (!folio) {
> +	if (!folio)
>  		folio = alloc_surplus_hugetlb_folio(h, gfp_mask, mpoli->nid,
> -						    nodemask);
> -	}
> +						    &alloc_nodemask);
> +
> +	if (!folio && unlikely(read_mems_allowed_retry(cpuset_mems_cookie)))
> +		goto retry;

[Severity: Medium]
Can pre-intersecting alloc_nodemask with cpuset_current_mems_allowed cause
non-gigantic surplus allocations to violate MPOL_BIND or use a stale nodemask
when cpuset_current_mems_allowed changes concurrently?

alloc_buddy_hugetlb_folio() is used for both gigantic and non-gigantic
surplus hugepage allocations. For non-gigantic hugepages,
alloc_surplus_hugetlb_folio() calls into __alloc_frozen_pages_noprof() ->
__alloc_pages_slowpath() -> check_retry_cpuset(), which expects ac->nodemask
to be the raw policy nodemask rather than a nodemask pre-intersected with an
earlier snapshot of cpuset_current_mems_allowed:

1. Suppose a VMA MPOL_BIND policy has *mpoli->nodemask = {0, 2} and
   current->mems_allowed is initially {0}, so alloc_nodemask is computed as
   {0}. If current->mems_allowed changes to {1, 2} while the thread enters
   __alloc_pages_slowpath(), check_retry_cpuset() sees that ac->nodemask ({0})
   no longer intersects current->mems_allowed ({1, 2}), even though the raw
   MPOL_BIND mask ({0, 2}) still intersects {1, 2} on node 2.
   check_retry_cpuset() then clears ac->nodemask = NULL and restarts the
   slowpath, which can allocate a folio on node 1 (allowed by the new cpuset
   {1, 2}, but forbidden by MPOL_BIND {0, 2}). Because folio is non-NULL,
   !folio && unlikely(read_mems_allowed_retry(cpuset_mems_cookie)) does not
   trigger a retry, and the folio from node 1 is returned.

2. In the MPOL_BIND check at the top of alloc_buddy_hugetlb_folio():

   	if (mpoli->mode == MPOL_BIND && mpoli->nodemask &&
   	    nodes_intersects(*mpoli->nodemask, alloc_nodemask))
   		nodes_and(alloc_nodemask, alloc_nodemask, *mpoli->nodemask);

   *mpoli->nodemask is read twice. If mpoli->nodemask points to a task
   mempolicy (current->mempolicy->nodes), a concurrent
   cpuset_change_task_nodemask() -> mpol_rebind_task() ->
   mpol_rebind_nodemask() can update pol->nodes between nodes_intersects()
   and nodes_and(), leaving alloc_nodemask empty.

3. For MPOL_DEFAULT (where nodemask was previously NULL), passing
   &alloc_nodemask (for example {0, 1}) into __alloc_frozen_pages_noprof()
   means that if current->mems_allowed changes to {1, 2} during
   __alloc_pages_slowpath(), check_retry_cpuset() sees a non-empty
   intersection on node 1 and loops in the slowpath with ac->nodemask still
   restricted to {0, 1}, preventing slowpath retries from considering newly
   allowed node 2.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1793850539.git.zhengqi.arch@bytedance.com?part=4