From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C01FC21B91D for ; Sun, 21 Dec 2025 18:28:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766341713; cv=none; b=IEiRzp/pMAkHErvKmPvHUoNfYiaUVreQhoPeI5gv0MyYeFn38ce7OkxJc+wP+FypM1skofoqEC2PHcPXX4C4LTB4RvnX3Fq/Igu8Mbr+tRQmYC1hNKqV6a7tAqDvouopqHWjxvF6MGCL9WhVhEAXmoB2S0ejJoYGql/XGm5965o= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766341713; c=relaxed/simple; bh=AkNI0IP65wDJFFRdu3MdafAv4S5yHYsoDfUw0WPLVwk=; h=Date:To:From:Subject:Message-Id; b=ZmpD76HVJoGkg147MxjWpXeZlR+3tt/FqO7V/WQkO1W4Yj6zvytSFIrG4LlbDgx2+SkOGLOqka4pYJkEYcoYT5LVlfz3oE6BuCs2KGw4TfWKODYKcbvuoF1YF75WI0bilB38HvidsZ4f8KnwAdAkd4hWNw3W/Ht3tkgHd8/RkbE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=Ol6Fyyso; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="Ol6Fyyso" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3D844C4CEFB; Sun, 21 Dec 2025 18:28:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1766341713; bh=AkNI0IP65wDJFFRdu3MdafAv4S5yHYsoDfUw0WPLVwk=; h=Date:To:From:Subject:From; b=Ol6Fyyso7tevHb4j9E88QHnaMJkuMMV6hrsP0mKc0h8ebRWTwF+BiKiHSo9BjxaV2 EybmsYdBR3R0F286dVnV3SjMEPdRgHm1mtrMtxVTrtCjDbgDlNkGD42wir6i9oMGOU NmszO4rFt7uVCT9sZdrzzjvxZSva+/l2Qrtm6Yq8= Date: Sun, 21 Dec 2025 10:28:32 -0800 To: mm-commits@vger.kernel.org,ziy@nvidia.com,vbabka@suse.cz,surenb@google.com,richard.weiyang@gmail.com,mhocko@suse.com,jackmanb@google.com,hannes@cmpxchg.org,david@redhat.com,gourry@gourry.net,akpm@linux-foundation.org From: Andrew Morton Subject: + page_alloc-allow-migration-of-smaller-hugepages-during-contig_alloc.patch added to mm-new branch Message-Id: <20251221182833.3D844C4CEFB@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: page_alloc: allow migration of smaller hugepages during contig_alloc has been added to the -mm mm-new branch. Its filename is page_alloc-allow-migration-of-smaller-hugepages-during-contig_alloc.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/page_alloc-allow-migration-of-smaller-hugepages-during-contig_alloc.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: Gregory Price Subject: page_alloc: allow migration of smaller hugepages during contig_alloc Date: Sun, 21 Dec 2025 07:46:56 -0500 We presently skip regions with hugepages entirely when trying to do contiguous page allocation. This will cause otherwise-movable 2MB HugeTLB pages to be considered unmovable, and makes 1GB gigantic page allocation less reliable on systems utilizing both. Commit 4d73ba5fa710 ("mm: page_alloc: skip regions with hugetlbfs pages when allocating 1G pages") skipped all HugePage containing regions because it can cause significant delays in 1G allocation (as HugeTLB migrations may fail for a number of reasons). Instead, if hugepage migration is enabled, consider regions with hugepages smaller than the target contiguous allocation request as valid targets for allocation. We optimize for the existing behavior by searching for non-hugetlb regions in a first pass, then retrying the search to include hugetlb only on failure. This allows the existing fast-path to remain the default case with a slow-path fallback to increase reliability. We only fallback to the slow path if a hugetlb region was detected, and we do a full re-scan because the zones/blocks may have changed during the first pass (and it's not worth further complexity). isolate_migrate_pages_block() has similar hugetlb filter logic, and the hugetlb code does a migratable check in folio_isolate_hugetlb() during isolation. The code servicing the allocation and migration already supports this exact use case. To test, allocate a bunch of 2MB HugeTLB pages (in this case 48GB) and then attempt to allocate some 1G HugeTLB pages (in this case 4GB) (Scale to your machine's memory capacity). echo 24576 > .../hugepages-2048kB/nr_hugepages echo 4 > .../hugepages-1048576kB/nr_hugepages Prior to this patch, the 1GB page reservation can fail if no contiguous 1GB pages remain. After this patch, the kernel will try to move 2MB pages and successfully allocate the 1GB pages (assuming overall sufficient memory is available). Also tested this while a program had the 2MB reservations mapped, and the 1GB reservation still succeeds. folio_alloc_gigantic() is the primary user of alloc_contig_pages(), other users are debug or init-time allocations and largely unaffected. - ppc/memtrace is a debugfs interface - x86/tdx memory allocation occurs once on module-init - kfence/core happens once on module (late) init - THP uses it in debug_vm_pgtable_alloc_huge_page at __init time Link: https://lkml.kernel.org/r/20251221124656.2362540-1-gourry@gourry.net Signed-off-by: Gregory Price Suggested-by: David Hildenbrand Link: https://lore.kernel.org/linux-mm/6fe3562d-49b2-4975-aa86-e139c535ad00@redhat.com/ Reviewed-by: Zi Yan Reviewed-by: Wei Yang Cc: Brendan Jackman Cc: Johannes Weiner Cc: Michal Hocko Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/page_alloc.c | 59 ++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55 insertions(+), 4 deletions(-) --- a/mm/page_alloc.c~page_alloc-allow-migration-of-smaller-hugepages-during-contig_alloc +++ a/mm/page_alloc.c @@ -7108,7 +7108,8 @@ static int __alloc_contig_pages(unsigned } static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, - unsigned long nr_pages) + unsigned long nr_pages, bool skip_hugetlb, + bool *skipped_hugetlb) { unsigned long i, end_pfn = start_pfn + nr_pages; struct page *page; @@ -7124,8 +7125,42 @@ static bool pfn_range_valid_contig(struc if (PageReserved(page)) return false; - if (PageHuge(page)) - return false; + /* + * Only consider ranges containing hugepages if those pages are + * smaller than the requested contiguous region. e.g.: + * Move 2MB pages to free up a 1GB range. + * Don't move 1GB pages to free up a 2MB range. + * + * This makes contiguous allocation more reliable if multiple + * hugepage sizes are used without causing needless movement. + */ + if (PageHuge(page)) { + unsigned int order; + + if (!IS_ENABLED(CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION)) + return false; + + if (skip_hugetlb) { + *skipped_hugetlb = true; + return false; + } + + page = compound_head(page); + order = compound_order(page); + if ((order >= MAX_FOLIO_ORDER) || + (nr_pages <= (1 << order))) + return false; + + /* + * Reaching this point means we've encounted a huge page + * smaller than nr_pages, skip all pfn's for that page. + * + * We can't get here from a tail-PageHuge, as it implies + * we started a scan in the middle of a hugepage larger + * than nr_pages - which the prior check filters for. + */ + i += (1 << order) - 1; + } } return true; } @@ -7168,7 +7203,10 @@ struct page *alloc_contig_pages_noprof(u struct zonelist *zonelist; struct zone *zone; struct zoneref *z; + bool skip_hugetlb = true; + bool skipped_hugetlb = false; +retry: zonelist = node_zonelist(nid, gfp_mask); for_each_zone_zonelist_nodemask(zone, z, zonelist, gfp_zone(gfp_mask), nodemask) { @@ -7176,7 +7214,9 @@ struct page *alloc_contig_pages_noprof(u pfn = ALIGN(zone->zone_start_pfn, nr_pages); while (zone_spans_last_pfn(zone, pfn, nr_pages)) { - if (pfn_range_valid_contig(zone, pfn, nr_pages)) { + if (pfn_range_valid_contig(zone, pfn, nr_pages, + skip_hugetlb, + &skipped_hugetlb)) { /* * We release the zone lock here because * alloc_contig_range() will also lock the zone @@ -7195,6 +7235,17 @@ struct page *alloc_contig_pages_noprof(u } spin_unlock_irqrestore(&zone->lock, flags); } + /* + * If we failed, retry the search, but treat regions with HugeTLB pages + * as valid targets. This retains fast-allocations on first pass + * without trying to migrate HugeTLB pages (which may fail). On the + * second pass, we will try moving HugeTLB pages when those pages are + * smaller than the requested contiguous region size. + */ + if (skip_hugetlb && skipped_hugetlb) { + skip_hugetlb = false; + goto retry; + } return NULL; } #endif /* CONFIG_CONTIG_ALLOC */ _ Patches currently in -mm which might be from gourry@gourry.net are mm-hugetlb-implement-movable_gigantic_pages-sysctl.patch page_alloc-allow-migration-of-smaller-hugepages-during-contig_alloc.patch