[RFC PATCH 25/45] mm: page_alloc: skip pageblock compatibility threshold in tainted SPBs

[Rik van Riel <riel@surriel.com>]

From: Rik van Riel <riel@fb.com>

Summary:
__rmqueue_smallest Pass 2b is the last resort before tainting a fresh
clean superpageblock: it walks MOVABLE sub-pageblock free chunks inside
already-tainted SPBs, calling try_to_claim_block() to relabel a movable
pageblock as the requested non-movable type. If Pass 2b fails, the
allocator falls through to Pass 3 and taints a clean SPB.

try_to_claim_block() guards the relabel with a 50% compatibility check:
free_pages + alike_pages must be at least pageblock_nr_pages/2. The
guard exists to protect a generic clean MOVABLE pageblock from being
relabeled when most of its pages are still in-use movable allocations.

Inside a tainted SPB the guard is harmful, not protective. The SPB has
already accepted fragmentation, and stranding a few in-use movable
pages inside a relabeled pageblock is dramatically cheaper than
tainting an entire clean SPB. bpftrace on a devvm under realistic load
caught the pathology directly: at the moment a clean SPB was tainted,
all 8 existing tainted SPBs had nr_free=0 (no whole free pageblocks),
collectively held ~21k movable free pages distributed across MOVABLE
pageblocks, and try_to_claim_block() had failed 29182 of 29228 calls
(99.84%) over the prior few minutes. Pass 2b was effectively unable
to absorb non-movable demand into the tainted pool.

Add a from_tainted_spb parameter to try_to_claim_block() and skip the
50% threshold when set. Pass 2b passes true (it walks SB_TAINTED lists
exclusively); __rmqueue_claim() passes false to preserve its existing
fragmentation-protection semantics.

Test Plan:
Devvm bpftrace setup at ~/spb-monitors/spb-taint-walk.bt watches
clean->tainted transitions in zone Normal and tracks
try_to_claim_block call/ok/fail counters. Before the change the fail
rate was 99.84% with periodic clean SPB taints under load. After the
change, expect the fail rate to drop sharply and the count of tainted
SPBs to plateau at the boot-recruited set.

Reviewers:

Subscribers:

Tasks:

Tags:

Signed-off-by: Rik van Riel <riel@surriel.com>
Assisted-by: Claude:claude-opus-4.7 syzkaller
---
 mm/page_alloc.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 493db531b869..67cc8165ab1f 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2776,7 +2776,8 @@ static struct page *claim_whole_block(struct zone *zone, struct page *page,
 		  int current_order, int order, int new_type, int old_type);
 static struct page *try_to_claim_block(struct zone *zone, struct page *page,
 		  int current_order, int order, int start_type,
-		  int block_type, unsigned int alloc_flags);
+		  int block_type, unsigned int alloc_flags,
+		  bool from_tainted_spb);
 
 static __always_inline
 struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
@@ -2941,7 +2942,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
 					page = try_to_claim_block(zone, page,
 						current_order, order,
 						migratetype, MIGRATE_MOVABLE,
-						0);
+						0, true);
 					if (!page)
 						continue;
 					trace_mm_page_alloc_zone_locked(
@@ -3420,11 +3421,17 @@ claim_whole_block(struct zone *zone, struct page *page,
  * not, we check the pageblock for constituent pages; if at least half of the
  * pages are free or compatible, we can still claim the whole block, so pages
  * freed in the future will be put on the correct free list.
+ *
+ * @from_tainted_spb: caller has already verified the block lives in a tainted
+ * superpageblock, where SPB-level fragmentation has already been accepted.
+ * Skip the per-pageblock compatibility threshold so we can absorb non-movable
+ * demand into the existing tainted SPB instead of tainting a fresh clean one.
  */
 static struct page *
 try_to_claim_block(struct zone *zone, struct page *page,
 		   int current_order, int order, int start_type,
-		   int block_type, unsigned int alloc_flags)
+		   int block_type, unsigned int alloc_flags,
+		   bool from_tainted_spb)
 {
 	int free_pages, movable_pages, alike_pages;
 	unsigned long start_pfn;
@@ -3480,8 +3487,14 @@ try_to_claim_block(struct zone *zone, struct page *page,
 	/*
 	 * If a sufficient number of pages in the block are either free or of
 	 * compatible migratability as our allocation, claim the whole block.
-	 */
-	if (free_pages + alike_pages >= (1 << (pageblock_order-1)) ||
+	 * The compatibility threshold protects clean MOVABLE pageblocks from
+	 * being relabeled when most of their pages are still in-use movable
+	 * allocations. Inside a tainted SPB the protection is unnecessary:
+	 * fragmentation has already been accepted at the SPB level, and
+	 * relabeling is much cheaper than tainting a fresh clean SPB.
+	 */
+	if (from_tainted_spb ||
+	    free_pages + alike_pages >= (1 << (pageblock_order-1)) ||
 			page_group_by_mobility_disabled) {
 		__move_freepages_block(zone, start_pfn, block_type, start_type);
 		set_pageblock_migratetype(pfn_to_page(start_pfn), start_type);
@@ -3721,7 +3734,8 @@ __rmqueue_claim(struct zone *zone, int order, int start_migratetype,
 
 			page = try_to_claim_block(zone, page, current_order,
 						  order, start_migratetype,
-						  fallback_mt, alloc_flags);
+						  fallback_mt, alloc_flags,
+						  false);
 			if (page) {
 				trace_mm_page_alloc_extfrag(page, order,
 					current_order, start_migratetype,
-- 
2.52.0